UDMG for Linux Installation
For the UDMG Web Transfer Client, please refer to UDMG Web Transfer Client for Linux Installation.
Upgrading Universal Data Mover Gateway
Upgrading UDMG refers to the increase of a currently installed pre-1.4.x Version, Release, or Modification level of UDMG (1.3.x, 1.2.x, 1.1.x, 1.0.x) to UDMG 1.4.x.
Note
If you are upgrading an installation of UDMG from any release prior to 1.3.0.0, you must uninstall the older version before installmysqling the new version.
The installation packages, binaries, services, and environment variables have changed and this does not allow for a standard upgrade.
Category | Prior releases | Release 1.3 |
---|---|---|
user and group | mft:mft | udmg:udmg |
binaries | /usr/local/bin | /opt/udmg/bin |
configuration files | /etc/mft | /opt/udmg/etc |
log files | /var/opt/udmg/logs | /var/opt/udmg/logs |
UDMG Admin UI assets | /opt/udmg/var/www/mft | /opt/udmg/var/www/udmg |
Services |
|
|
Environment variables |
|
|
It may be required to modify the work and data directories ownership or access rights and to update UDMG Server transfer rules to use paths that are accessible by the 'udmg' user.
The configuration files must be reviewed and compared between the old and new locations.
Special attention is required for the AESpassphrase parameter for UDMG Server. It must be the path for the file that was used by the previous release and must be accessible by the new service user. It is recommended to set an absolute path in the configuration file.
Pre-Installation / Upgrade Backups
The installation process overwrites the current files (exception: the configuration files are kept), this may affect your modifications.
Backing up the configuration files will optimize the time it takes you to get up and running after installing or upgrading.
/opt/udmg/etc/udmg/nginx/udmg.conf
/opt/udmg/etc/udmg/agent/client.toml
/opt/udmg/etc/udmg/agent/server.toml
/opt/udmg/etc/udmg/auth-proxy/config.toml
/opt/udmg/etc/udmg/web-transfer/config.toml
/opt/udmg/etc/udmg-server/server.ini
After upgrading RPM or DEB packages, review the new configuration file templates (with the extension .rpmnew or .dpk-new) and edit the current configuration files to add new parameters or remove deprecated parameters.
Release migration
The UDMG release version is stored in the database to ensure the data structure is compatible with the version of the UDMG components.
After upgrading the component binaries and before starting the UDMG server it is required to perform the release migration step.
The udmg-server "migrate" command handles the necessary database updates and the setting of the internal version.
$ /opt/udmg/bin/udmg-server migrate --help Usage: udmg-server [OPTIONS] migrate [migrate-OPTIONS] [version] Help Options: -h, --help Show this help message [migrate command options] -c, --config= The configuration file to use -d, --dry-run Simulate the migration but does not commit the changes -l, --list List Migrations -f, --file= Writes the migration commands into a file instead of sending them to the database -v, --verbose Show verbose debug information. Can be repeated to increase verbosity [migrate command arguments] version: The version to which the database should be migrated
The configuration file is the one that will be used for the server mode, with the parameter for accessing the target database.
To get the list of supported target version, use the list parameter. The last version is the release version of the udmg-server.
$ /opt/udmg/bin/udmg-server migrate -c /opt/udmg/etc/udmg-server/server.ini --list | tail -n 3 1.3.0 1.4.0 1.4.1
It is recommended to set the verbose parameter (3 times) to follow the progress.
$ /opt/udmg/bin/udmg-server migrate -c /opt/udmg/etc/udmg-server/server.ini -vvv 1.4.1 [INFO ] Migration: Starting upgrade migration... [INFO ] Migration: Applying migration 'Bump database version to 1.4.1'
After the migration, the services for UDMG components can be started.
Upgrading Universal Data Mover Gateway for Linux
Upgrading with Linux software packages
Step 1 | Contact your Stonebranch representative or the Customer Support to receive the software package for the intended operating system. |
---|---|
Step 2 | Perform the recommended UDMG for Linux Installation#backup of configuration files. |
Step 3 | Stop the components services. The exact steps depend on the system architecture and the deployed components, for example:
|
Step 4 | Upgrade the UDMG packages (RPM or DEB), for example:
|
Step 5 | Review the component configuration files. Note that new configuration file templates (with the extension .rpmnew or .dpk-new) that contains all the allowed parameters are added during the software package upgrade. |
Step 6 | Perform the release migration. |
Step 7 | Start the components services. The exact steps depend on the system architecture and the deployed components, for example:
|
Upgrading a manual installation
Step 1 | Contact your Stonebranch representative or the Customer Support to receive the software package for the intended operating system. |
---|---|
Step 2 | Perform the recommended UDMG for Linux Installation#backup of configuration files. |
Step 3 | Stop the components services. The exact steps depend on the system architecture and the deployed components, for example:
|
Step 4 | Replace the component binaries
Change ownership/permissions on new files:
Upgrade the Admin UI:
|
Step 5 | Review the component configuration files. Refer to each component installation section below for the list of parameters. |
Step 6 | Perform the release migration. |
Step 7 | Start the components services. The exact steps depend on the system architecture and the deployed components, for example:
|
Installing and Configuring the Components
Installing with Linux software packages
Step 1 | Contact your Stonebranch representative or the Customer Support to receive the software package for the intended operating system. |
---|---|
Step 2 | Install the UDMG packages (RPM or DEB), for example:
|
Step 3 | Review the component configuration files. |
Step 4 | Start the components services. The exact steps depend on the system architecture and the deployed components, for example:
|
Performing a manual installation
UDMG Admin UI
The following steps require root privilege, make sure that you have the correct access before continuing.
Extract the distribution file for UDMG Admin UI, under the directory web server root directory, see the NGINX Server configuration.
#
unzip -d /opt/udmg/var/www/udmg/ udmg-admin-ui-<VERSION>.zipsudo
- The zip file can now be deleted.
- Validate that the service is working properly with the 'curl' command:
# curl http://localhost:80 -I
HTTP/1.1 200 OK
Server: nginx/1.21.6
Date: Mon, 06 Jun 2022 17:33:19 GMT
Content-Type: text/html
Content-Length: 7788
Last-Modified: Fri, 03 Jun 2022 14:07:05 GMT
Connection: keep-alive
ETag: "629a1589-1e6c"
Accept-Ranges: bytes
UDMG User setup
Create a dedicated user for running the UDMG modules and to be the owner of the files that will be transferred by UDMG.
#
groupadd udmgsudo
#
useradd -g udmg udmgsudo
UDMG Server
Create the configuration file /opt/udmg/etc/udmg-server/server.ini with the following parameters:
#
mkdir -p /opt/udmg/etc/udmg-serversudo
#
vi /opt/udmg/etc/udmg-server/server.inisudo
Note
The lines starting with a semicolon ';' or a hash '#' are comments, describing the option or showing the default value.
The parameters must be adapted to your environment, in particular:
- global section: GatewayHome
- log section: LogLevel, LogTo, LogPath
- admin section: Host, Port
- database section: Type, Address, Name, User, Password
[global]
; The name given to identify this gateway instance. If the the database is shared between multiple gateways, this name MUST be unique across these gateways.
GatewayName = udmg
; Default OS permission for created files
; FilePermissions = 770
; Default OS permission for created directories
; DirPermissions = 770
[paths]
; The root directory of the gateway. By default, it is the working directory of the process.
GatewayHome = /home/udmg
; The directory for all incoming files.
; DefaultInDir = in
; The directory for all outgoing files.
; DefaultOutDir = out
; The directory for all running transfer files.
; DefaultTmpDir = tmp
[log]
; All messages with a severity above this level will be logged. Possible values are TRACE, DEBUG, INFO, WARNING, ERROR and CRITICAL.
Level = DEBUG
; The path to the file where the logs must be written. Special values 'stdout' and 'syslog' log respectively to the standard output and to the syslog daemon.
; LogTo = stdout
; If LogTo is set on 'syslog', the logs will be written to this facility.
; SyslogFacility = local0
; The directory for the log files of the local servers, partners, and transfers. No default, if not provided then the detailed log feature is disabled.
; LogPath =
/var/opt/udmg/logs
[admin]
; The address used by the admin interface.
Host = 0.0.0.0
; The port used by the admin interface. If the port is 0, a free port will automatically be chosen.
Port = 18080
; Path of the TLS certificate for the admin interface.
; TLSCert =
; Path of the key of the TLS certificate.
; TLSKey =
; API rate limiter: number of allowed requests per client IP, per second. After that HTTP code 429 is returned. Disabled if 0 or not provided.
; RateLimit = 0
[database]
; Name of the RDBMS used for the gateway database. Possible values: sqlite, mysql, postgresql, oracle, mssql
Type = postgresql
; Address of the database
Address = localhost
; The name of the database
Name = udmg
; The name of the gateway database user
User = udmg_user
; The password of the gateway database user
Password = udmg_password
; Path of the database TLS certificate file.
; TLSCert =
; Path of the key of the TLS certificate file.
; TLSKey =
; The path to the file containing the passphrase used to encrypt account passwords using AES
; AESPassphrase = passphrase.aes
[controller]
; The frequency at which the database will be probed for new transfers
; Delay = 5s
; The maximum number of concurrent incoming transfers allowed on the gateway (0 = unlimited).
; MaxTransferIn = 0
; The maximum number of concurrent outgoing transfers allowed on the gateway (0 = unlimited).
; MaxTransferOut = 0
[sftp]
; Set to true to allow legacy and weak cipher algorithms: 3des-cbc, aes128-cbc, arcfour, arcfour128, arcfour256
; AllowLegacyCiphers = false
Note
AESPassphrase file is generated on first run if it does not exist. It is recommended to set an absolute path, otherwise it is created in the current directory.
Make sure to verify the file location during an upgrade and to have a backup. Without the correct AESPassphrase file, the passwords, the keys and the certificates will not be usable.
The use of an incorrect passphrase file is reported with the error:
cannot decrypt password: cipher: message authentication failed
- Install the binaries under /opt/udmg/bin
:
#
install -m 755 udmg-client /opt/udmg/binsudo
#
install -m 755 udmg-server /opt/udmg/binsudo
UDMG Authentication Proxy
Create a directory under
/etc/udmg/:
#
mkdir -p /opt/udmg/etc/udmgsudo
- Create a configuration file for the service:
#
vi /opt/udmg/etc/udmg/auth_proxy/config.tomlsudo
# Proxy Configuration
[proxy]
# Port, default "5000"
port = "5000"
# Network interface, default "0.0.0.0"
inet = "127.0.0.1"
# Enable recover on panic, default true, should be true for production environment
recover = true
# Enable Cross-Origin Resource Sharing (CORS), should be true for production environment
cors = true
#
CORS:
List of origins that may access the resource. Optional. Default value "*"# domain
= "*"
# Enable Request Track ID, default truetracker = true
# Enable Request Logger, default true
logger = true
# Rate Limit IP Request over 1 second, default 0 (unlimited)
limit = 0
# Enable the Prometheus Metric Endpoint '/metric', default false
metrics = false
# Enable CSRF tokencsrf = false
#
# Service 'local' with direct authentication on the
UDMG Server
[service.local]
# UDMG Server Listen Protocol
protocol = "http"
[[service.local.targets]]
# UDMG Server Hostname or IP
hostname = "localhost"
#
UDMG Server
Portport = 18080
Please refer to Authentication Methods for the LDAP and SSO authentication options.
- Install the binary under
/opt/udmg/bin:
#
install -m 755 udmg-auth-proxy /opt/udmg/binsudo
UDMG Agent Proxy
Create a directory under /opt/udmg/etc/udmg
:
#
mkdir -p /opt/udmg/etc/udmg/agent/sudo
- Install the binaries under
/opt/udmg/bin:
#
install -m 755 sudo
udmg-agent-client
/opt/udmg/bin#
install -m 755 sudo
udmg-agent-
server /opt/udmg/bin
Agent Proxy Server Configuration
Generate a SSH Key for the service:
# ssh-keygen -t rsa -q -N "" -f /opt/udmg/etc/udmg/agent/agent
- Change the agent key permissions:
# chmod 755 /opt/udmg/etc/udmg/agent/agent /opt/udmg/etc/udmg/agent/agent.pub
- Create a configuration file as /opt/udmg/etc/udmg/agent/
agent.toml:
#
vi /opt/udmg/etc/udmg/agent/agent.tomlsudo
[agent]
# UDMG Agent Proxy Hostname or IP, and port
hostname = "0.0.0.0"
port = "2222"
# path to the SSH private key file
ssh_key = "/opt/udmg/etc/udmg/agent/agent"
# path to the SSH public key file
ssh_key_pub = "/opt/udmg/etc/udmg/agent/agent.pub"
# Agent Service User and password
username = "mft"
password = "61ee8b5601a84d5154387578466c8998848ba089"
The password key will be used for the client authentication.
Agent Proxy Client Configuration
Create a configuration file as
/etc/
udmg
/agent_proxy/client.toml:
#
vi /opt/udmg/etc/udmg/agent/client.tomlsudo
[client]
# Target UDMG Agent Proxy Hostname or IP, and port
hostname = "localhost"
port = "2222"
#
UDMG
Agent Service User and passwordusername = "mft"
password = "61ee8b5601a84d5154387578466c8998848ba089"
# Default TTL to Connection Retry
ttl="5s"
[client.api]
# UDMG Agent Client Admin API
port="2280"
[gateway]
#
UDMG Server
Hostname or IP, and port hostname = "localhost"
port = "18080"
# UDMG Server Username/Password
username = "admin"
password = "admin_password"
The password key will be used for the client authentication.
Setup the Systemd Services
UDMG Server
Create a new service definition:
#
vi /etc/systemd/system/udmg-server.servicesudo
[Unit]
Description=UDMG Server
[Service]
Type=simple
User=udmg
Group=udmg
WorkingDirectory=/home/udmg
ExecStart=/bin/sh -c '/usr/local
/bin/udmg-server server -c /opt/udmg/etc/udmg-server/server.ini'
Restart=on-failure
[Install] WantedBy=multi-user.target
- Enable the new service:
#
systemctl enable sudo
udmg-server
.serviceCreated symlink /etc/systemd/system/multi-user.target.wants/
udmg-server
.service → /etc/systemd/system/udmg-server
.service.
- Start the service and check the status:
# systemctl start
udmg-server
# systemctl status
udmg-server
●
.service - UDMG serverudmg-server
Loaded: loaded ( /etc/systemd/system/
udmg-server
.service; enabled; vendor preset: disabled)Active: active (running) since Tue 2022-06-07 16:43:16 -03; 10s ago
Main PID: 24888 (
)udmg-server
Tasks: 6 (limit: 3509)
CPU: 11ms
CGroup: /system.slice/
udmg-server
.service└─24888 /opt/udmg/bin/
udmg-server
server -c /opt/udmg/etc/udmg-server/server.ini
Make sure that the listen port and network interface is reachable by UDMG Authentication Proxy and UDMG Agent Client.
UDMG Authentication Proxy
Create a new service definition:
#
vi /etc/systemd/system/sudo
.serviceudmg-auth-proxy
[Unit]
Description=UDMG Auth Proxy server
[Service]
Type=simple
User=udmg
Group=udmg
WorkingDirectory=/home/udmg
Environment="UDMG_AUTH_PROXY_CONFIG=/opt/udmg/etc/udmg/auth_proxy/config.toml"
ExecStart=/bin/sh -c 'exec /opt/udmg/bin/
'udmg-auth-proxy
Restart=on-failure
[Install]
WantedBy=multi-user.target
- Enable the new service:
#
systemctl enable sudo
.serviceudmg-auth-proxy
Created symlink /etc/systemd/system/multi-user.target.wants/
.service → /etc/systemd/system/udmg-auth-proxy
.service.udmg-auth-proxy
- Start the service and check the status:
# systemctl start
udmg-auth-proxy
# systemctl status
udmg-auth-proxy
●
.service - UDMG Auth Proxy serverudmg-auth-proxy
Loaded: loaded ( /etc/systemd/system/
.service; enabled; vendor preset: disabled)udmg-auth-proxy
Active: active (running) since Tue 2022-06-07 16:58:48 -03; 21s ago
Main PID: 25008 (
udmg-auth-proxy
)Tasks: 3 (limit: 3509)
CPU: 4ms
CGroup: /system.slice/
.serviceserviceudmg-auth-proxy
└─25008 /opt/udmg/bin/
udmg-auth-proxy
Make sure that the listen port and network interface is reachable by NGINX Server.
UDMG Agent Proxy
Agent Proxy Server Service
Create a new service definition:
#
vi /etc/systemd/system/udmg-agent-server.servicesudo
[Unit]
Description=UDMG Agent Proxy server
[Service]
Type=simple
User=udmg
Group=udmg
WorkingDirectory=/home/udmg
Environment="UDMG_AGENT_CONFIG=/opt/udmg/etc/udmg/agent/server.toml"
ExecStart=/bin/sh -c 'exec /opt/udmg/bin/udmg-agent-server'
Restart=on-failure
[Install]
WantedBy=multi-user.target
- Enable the new service:
#
systemctl enable udmg-agent-serversudo
Created symlink /etc/systemd/system/multi-user.target.wants/udmg-agent-server.service → /etc/systemd/system/udmg-agent-server.service.
- Start the service and check the status:
# systemctl start udmg-agent-server
# systemctl status udmg-agent-server
● udmg-agent-server.service - UDMG Agent Proxy Server
Loaded: loaded ( /etc/systemd/system/udmg-agent-server.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2022-06-07 16:26:53 -03; 2s ago
Main PID: 25444 (udmg-agent-server)
Tasks: 5 (limit: 3509)
CPU: 5ms
CGroup: /system.slice/udmg-agent-server.service
└─25444 /opt/udmg/bin/udmg-agent-server
Jun 07 16:26:53 localhost.localdomain systemd[1]: Started UDMG Agent Proxy Server.
Jun 07 16:26:53 localhost.localdomain sh[25444]: level=info TS=2022-06-07T19:26:53.624296821Z HostKey=Ok Path=/data/agent
Be sure that the listen port and network interface is reachable by UDMG Agent Client .
Agent Proxy Client Service
Create a new service definition:
#
vi /etc/systemd/system/udmg-agent-client.servicesudo
[Unit]
Description=UDMG Agent Proxy Client
[Service]
Type=simple
User=udmg
Group=
udmg
WorkingDirectory=/home/
udmg
Environment="UDMG_AGENT_CONFIG=/opt/udmg/etc/udmg/agent/client.toml"
ExecStart=/bin/sh -c 'exec /opt/udmg/bin/
udmg-
agent-client'Restart=on-failure
[Install]
WantedBy=multi-user.target
- Enable the new service:
#
systemctl enable sudo
udmg-agent-client
.serviceCreated symlink /etc/systemd/system/multi-user.target.wants/
udmg-agent-client
.service → /etc/systemd/system/udmg-agent-client
.service.
- Start the service and check the status:
# systemctl start
udmg-agent-client
# systemctl status
udmg-agent-client
●
.service - UDMG Agent Proxy Clientudmg-agent-client
Loaded: loaded ( /etc/systemd/system/
udmg-agent-client
.service; enabled; vendor preset: disabled)Active: active (running) since Tue 2022-06-07 17:26:53 -03; 2s ago
Main PID: 25445 (
udmg-agent-client
)Tasks: 5 (limit: 3509)
CPU: 6ms
CGroup: /system.slice/
udmg-agent-client
.service└─25445 /opt/udmg/bin/
udmg-agent-client
Jun 07 17:26:53 localhost.localdomain systemd[1]: Started UDMG Agent Proxy Client.
Jun 07 17:26:53 localhost.localdomain sh[25445]: level=info TS=2022-06-07T20:26:53.624296821Z Servers=[]
Ports Configuration
Using UDMG with SELinux
Security-Enhanced Linux (SELinux) is enabled by default on modern RHEL and CentOS servers. Each operating system object (process, file descriptor, file, etc.) is labeled with an SELinux context that defines the permissions and operations the object can perform. In RHEL 6.6/CentOS 6.6 and later, NGINX is labeled with the httpd_t
context.
When SELinux is enabled, the UDMG Admin UI will show "403 access denied" and "404 page not found" errors on the landing page and permission errors are reported in the NGINX log files:
...
2023/09/19 12:51:38 [error] 108236#108236: *1 "/opt/udmg/var/www/udmg/index.html" is forbidden (13: Permission denied), client: 127.0.0.1, server: localhost, request: "GET / HTTP/1.1", host: "localhost
...
- Verify whether SELinux is enforced with the
getenforce
command. A result of "Enforcing" means that is is enabled and that the steps below are required.
$ getenforce
Enforcing
- Modify the file labels so that NGINX (as a process labeled with the
httpd_t
context) can access the configuration files.
#
restorecon /etc/nginx/conf.d/*sudo
- Modify the file labels so that NGINX (as a process labeled with the
httpd_t
context) can access the UDMG Admin UI asset files.
#
semanage fcontext -a -t httpd_sys_content_t '/opt/udmg/var/www(/.*)?'sudo
#
restorecon -Rv /opt/udmg/var/wwwsudo
- Allow NGINX to reverse proxy through the UDMG Authentication Proxy by setting the
httpd_can_network_connect
boolean.
#
setsebool -P httpd_can_network_connect 1sudo
References
This document references the following documents.
Name | Location |
---|---|
Systemd | |
NGINX with SELinux | |
PostgreSQL Client Authentication | |
PostgreSQL Password Authentication |