Installing and Configuring the Components

UDMG Admin UI

The following steps require root privilege, be sure that you have the correct access before to continue.

Extract the distribution file for UDMG Admin UI, under the directory web server root directory, see the NGINX Service configuration above.

# unzip -d /srv/www/mft/ udmg_admin_ui-<VERSION>.zip

Validate that the service is working properly:

# curl http://localhost:8080 -I
HTTP/1.1 200 OK
Server: nginx/1.21.6
Date: Mon, 06 Jun 2022 17:33:19 GMT
Content-Type: text/html
Content-Length: 7788
Last-Modified: Fri, 03 Jun 2022 14:07:05 GMT
Connection: keep-alive
ETag: "629a1589-1e6c"
Accept-Ranges: bytes

UDMG User setup

Create a dedicated user for running the UDMG modules and to be the owner of the files that will be transferred by UDMG.

# useradd mft

UDMG Server

Create the configuration file /etc/mft/waarp_gateway/server.ini with the following parameters:

# mkdir -p /etc/mft/waarp_gateway
# vi /etc/mft/waarp_gateway/server.ini

[global]
; The name given to identify this gateway instance. If the the database is shared between multiple gateways, this name MUST be unique across these gateways.
GatewayName = mft_waarp_gateway

; Default OS permission for created files
; FilePermissions = 700

; Default OS permission for created directories
; DirPermissions = 750

[paths]
; The root directory of the gateway. By default, it is the working directory of the process.
GatewayHome = /home/mft

; The directory for all incoming files.
; DefaultInDir = in

; The directory for all outgoing files.
; DefaultOutDir = out

; The directory for all running transfer files.
; DefaultTmpDir = tmp

[log]
; All messages with a severity above this level will be logged. Possible values are DEBUG, INFO, WARNING, ERROR and CRITICAL.
Level = DEBUG

; The path to the file where the logs must be written. Special values 'stdout' and 'syslog' log respectively to the standard output and to the syslog daemon
; LogTo = stdout

; If LogTo is set on 'syslog', the logs will be written to this facility.
; SyslogFacility = local0

[admin]
; The address used by the admin interface.
Host = 0.0.0.0

; The port used by the admin interface. If the port is 0, a free port will automatically be chosen.
Port = 18080

; Path of the TLS certificate for the admin interface.
; TLSCert =

; Path of the key of the TLS certificate.
; TLSKey =

[database]
; Name of the RDBMS used for the gateway database. Possible values: sqlite, mysql, postgresql
Type = postgresql

; Address of the database
Address = localhost

; The name of the database
Name = mft_waarp_gateway

; The name of the gateway database user
User = mft_waarp_gateway_user

; The password of the gateway database user
Password = mft_waarp_gateway_password

; Path of the database TLS certificate file.
; TLSCert =

; Path of the key of the TLS certificate file.
; TLSKey =

; The path to the file containing the passphrase used to encrypt account passwords using AES
; AESPassphrase = passphrase.aes

[controller]
; The frequency at which the database will be probed for new transfers
Delay = 300s

; The maximum number of concurrent incoming transfers allowed on the gateway (0 = unlimited).
; MaxTransferIn = 0

; The maximum number of concurrent outgoing transfers allowed on the gateway (0 = unlimited).
; MaxTransferOut = 0

[sftp]
; Set to true to allow legacy and weak cipher algorithms: 3des-cbc, aes128-cbc, arcfour, arcfour128, arcfour256
; AllowLegacyCiphers = false

Install the binaries under /usr/local/bin:

# install -m 755 waarp-gatewayd /usr/local/bin
# install -m 755 waarp-gateway /usr/local/bin

UDMG Authentication Proxy

Create a directory under /etc/mft/:

# mkdir -p /etc/mft/auth_proxy

Create a configuration file for the service:

# vi /etc/mft/auth_proxy/config.toml

# Proxy Configuration
[proxy]
# Port, default "5000"
port = "5000"
# Network interface, default "0.0.0.0"
inet = "127.0.0.1"
# Enable recover on panic, default true, should be true for production environment
recover = true
# Enable Cross-Origin Resource Sharing (CORS), should be true for production environment
cors = true
# Enable Request Track ID, default true
tracker = true
# Enable Request Logguer, default true
logger = true
# Rate Limit IP Request over 1 second, default 0 (unlimited)
limit = 0
# Enable the Prometheus Metric Endpoint '/metric', default false
metrics = false

# Service 'local' with direct authentication on the waarp gateway
[service.local]
# MFT Waarp Gateway Listen Protocol
protocol = "http"

[[service.local.targets]]
# MFT Waarp Gateway Hostname or IP
hostname = "localhost"
# MFT Waarp Gateway Port
port = 18080

# Service 'mft' with direct authentication on the waarp gateway
[service.mft]
# MFT Waarp Gateway Listen Protocol
protocol = "http"

[[service.mft.targets]]
# MFT Waarp Gateway Hostname or IP
hostname = "localhost"
# MFT Waarp Gateway Port
port = 18080

Install the binary under /usr/local/bin:

# install -m 755 mft_auth_proxy_server /usr/local/bin

Configuration for LDAP Authentication

The UDMG Authentication Proxy is capable to use a LDAP Service to authenticate users for UDMG Admin UI:

# vi /etc/mft/auth_proxy/config.toml

# Proxy Configuration
[proxy]
# Port, default "5000"
port = "5000"
# Network interface, default "0.0.0.0"
inet = "127.0.0.1"
# Enable recover on panic, default true, should be true for production environment
recover = true
# Enable Cross-Origin Resource Sharing (CORS), should be true for production environment
cors = true
# Enable Request Track ID, default true
tracker = true
# Enable Request Logguer, default true
logger = true
# Rate Limit IP Request over 1 second, default 0 (unlimited)
limit = 0
# Enable the Prometheus Metric Endpoint '/metric', default false
metrics = false

# Service 'mft' with LDAP Authentication
[service.mft]
# MFT Waarp Gateway connection protocol (http or https)
protocol = "http"
# This is breaking glass option for admins,
# the users in the admins list are authenticated directly on the MFT service, not with LDAP
admins = ["admin"]

[[service.mft.targets]]
# MFT Waarp Gateway Hostname or IP
hostname = "localhost"
# MFT Waarp Gateway Port
port = 18080

# Credentials for the synchronisation from LDAP to MFT service
# user must have permission to create/update waarp gateway users
[service.mft.credential]
username = "ldap_sync"
password = "ldap_password"

# LDAP Configuration
[service.mft.auth.ldap]
# LDAP Server DC with OU
dn = "ou=users,dc=stonebranch,dc=com"
# LDAP Server FQDN or IP
hostname = "myldap.server.fqdn.com"
# LDAP Server Port
port = "1389"

The LDAP replication requires a user with permission for creating and updating users. For example to create the 'ldap_sync' user with the command line interface:

waarp_gateway user add -u ldap_sync -p ldap_password -r 'U=rw'

In case of successful authentication on the LDAP, the user is created with default read permission in the internal UDMG database if it does not exist. Otherwise the credentials are updated in the database to allow for authentication on the REST and CLI interfaces.

UDMG Agent Proxy

Create a directory under /etc/mft:

# mkdir -p /etc/mft/agent_proxy

Install the binaries under /usr/local/bin:

# install -m 755 mft_agent_proxy_client /usr/local/bin
# install -m 755 mft_agent_proxy_server /usr/local/bin

Agent Configuration

Generate a SSH Key for the service:

# ssh-keygen -t rsa -q -N "" -f /etc/mft/agent_proxy/agent
# ssh-keygen -t rsa -q -N "" -f /etc/mft/agent_proxy/client

Change the agent key permissions:

# chmod 755 /etc/mft/agent_proxy/agent /etc/mft/agent_proxy/agent.pub

Create a configuration file as /etc/mft/agent_proxy/agent.toml:

# vi /etc/mft/agent_proxy/agent.toml

[agent]
# MFT Agent Proxy Hostname or IP, and port
hostname = "0.0.0.0"
port = "2222"
# path to the SSH private key file
ssh_key = "/etc/mft/agent_proxy/agent"
# path to the SSH public key file
ssh_key_pub = "/etc/mft/agent_proxy/agent.pub"

# Agent Service User and password
username = "mft"
password = "61ee8b5601a84d5154387578466c8998848ba089"

The password key will be used for the client authentication.

Client Configuration

Create a configuration file as /etc/mft/agent_proxy/client.toml:

# vi /etc/mft/agent_proxy/client.toml

[client]
# Target MFT Agent Proxy Hostname or IP, and port
hostname = "localhost"
port = "2222"

# path to the SSH private key file
ssh_key = "/etc/mft/agent_proxy/client"
# path to the SSH public key file
ssh_key_pub = "/etc/mft/agent_proxy/client.pub"

# Agent Service User and password
username = "mft"
password = "61ee8b5601a84d5154387578466c8998848ba089"

# Default TTL to Connection Retry
ttl="5s"

[client.api]
# Administrative API port
port="2280"

[gateway]
# MFT Waarp Gateway Hostname or IP, and port
hostname = "localhost"
port = "18080"
# MFT Waarp Gateway Username/Password
username = "admin"
password = "admin_password"

The password key will be used for the client authentication.

Setup the Systemd Services

UDMG Server

Create a new service definition:

# vi /etc/systemd/system/mft_waarp_gateway.service

[Unit]
Description=MFT Waarp Gateway server

[Service]
Type=simple
User=mft
Group=mft
WorkingDirectory=/home/mft
ExecStart=/bin/sh -c 'exec /usr/local/bin/waarp-gatewayd server -c /etc/mft/waarp_gateway/server.ini'
Restart=on-failure

[Install]
WantedBy=multi-user.target

Enable the new service:

# systemctl enable mft_waarp_gateway.service
Created symlink /etc/systemd/system/multi-user.target.wants/mft_waarp_gateway.service → /etc/systemd/system/mft_waarp_gateway.service.

Start the service and check the status:

# systemctl start mft_waarp_gateway
# systemctl status mft_waarp_gateway
● mft_waarp_gateway.service - MFT Waarp Gateway server
Loaded: loaded ( /etc/systemd/system/mft_waarp_gateway.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2022-06-07 16:43:16 -03; 10s ago
Main PID: 24888 (waarp-gatewayd)
Tasks: 6 (limit: 3509)
CPU: 11ms
CGroup: /system.slice/mft_waarp_gateway.service
└─24888 /usr/local/bin/waarp-gatewayd server -c /etc/mft/waarp_gateway/server.ini

Be sure that the listen port and network interface is reachable by UDMG Authentication Proxy and UDMG Agent Client.

UDMG Authentication Proxy

Create a new service definition:

# vi /etc/systemd/system/mft_auth_proxy.service

[Unit]
Description=MFT Auth Proxy server

[Service]
Type=simple
User=mft
Group=mft
WorkingDirectory=/home/mft
Environment="MFT_AUTH_PROXY_CONFIG=/etc/mft/auth_proxy/config.toml"
ExecStart=/bin/sh -c 'exec /usr/local/bin/mft_auth_proxy_server'
Restart=on-failure

[Install]
WantedBy=multi-user.target

Enable the new service:

# systemctl enable mft_auth_proxy.service
Created symlink /etc/systemd/system/multi-user.target.wants/mft_auth_proxy.service → /etc/systemd/system/mft_auth_proxy.service.

Start the service and check the status:

# systemctl start mft_auth_proxy
# systemctl status mft_auth_proxy
● mft_auth_proxy.service - MFT Auth Proxy server
Loaded: loaded ( /etc/systemd/system/mft_auth_proxy.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2022-06-07 16:58:48 -03; 21s ago
Main PID: 25008 (mft_auth_proxy_)
Tasks: 3 (limit: 3509)
CPU: 4ms
CGroup: /system.slice/mft_auth_proxy.serviceservice
└─25008 /usr/local/bin/mft_auth_proxservicey_server

Be sure that the listen port and network interface is reachable by NGINX Server.

UDMG Agent Proxy

Agent Proxy Server Service

Create a new service definition:

# vi /etc/systemd/system/mft_agent_proxy_server.service

[Unit]
Description=MFT Agent Proxy Server

[Service]
Type=simple
User=mft
Group=mft
WorkingDirectory=/home/mft
Environment="MFT_AGENT_PROXY_CONFIG=/etc/mft/agent_proxy/agent.toml"
ExecStart=/bin/sh -c 'exec /usr/local/bin/mft_agent_proxy_server'
Restart=on-failure

[Install]
WantedBy=multi-user.target

Enable the new service:

# systemctl enable mft_agent_proxy_server.service
Created symlink /etc/systemd/system/multi-user.target.wants/mft_agent_proxy_server.service → /etc/systemd/system/mft_agent_proxy_server.service.

Start the service and check the status:

# systemctl start mft_agent_proxy_server
# systemctl status mft_agent_proxy_server
● mft_agent_proxy_server.service - MFT Agent Proxy Server
Loaded: loaded ( /etc/systemd/system/mft_agent_proxy_server.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2022-06-07 16:26:53 -03; 2s ago
Main PID: 25444 (mft_agent_proxy)
Tasks: 5 (limit: 3509)
CPU: 5ms
CGroup: /system.slice/mft_agent_proxy_server.service
└─25444 /usr/local/bin/mft_agent_proxy_server

Jun 07 16:26:53 localhost.localdomain systemd[1]: Started MFT Agent Proxy Server.
Jun 07 16:26:53 localhost.localdomain sh[25444]: level=info TS=2022-06-07T19:26:53.624296821Z HostKey=Ok Path=/data/agent

Be sure that the listen port and network interface is reachable by UDMG Agent Client .

Agent Proxy Client Service

Create a new service definition:

# vi /etc/systemd/system/mft_agent_proxy_client.service

[Unit]
Description=MFT Agent Proxy Client

[Service]
Type=simple
User=mft
Group=mft
WorkingDirectory=/home/mft
Environment="MFT_AGENT_PROXY_CONFIG=/etc/mft/agent_proxy/client.toml"
ExecStart=/bin/sh -c 'exec /usr/local/bin/mft_agent_proxy_client'
Restart=on-failure

[Install]
WantedBy=multi-user.target

Enable the new service:

# systemctl enable mft_agent_proxy_client.service
Created symlink /etc/systemd/system/multi-user.target.wants/mft_agent_proxy_client.service → /etc/systemd/system/mft_agent_proxy_client.service.

Start the service and check the status:

# systemctl start mft_agent_proxy_client
# systemctl status mft_agent_proxy_client
● mft_agent_proxy_client.service - MFT Agent Proxy Client
Loaded: loaded ( /etc/systemd/system/mft_agent_proxy_client.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2022-06-07 17:26:53 -03; 2s ago
Main PID: 25445 (mft_agent_proxy)
Tasks: 5 (limit: 3509)
CPU: 6ms
CGroup: /system.slice/mft_agent_proxy_client.service
└─25445 /usr/local/bin/mft_agent_proxy_client

Jun 07 17:26:53 localhost.localdomain systemd[1]: Started MFT Agent Proxy Server.
Jun 07 17:26:53 localhost.localdomain sh[25445]: level=info TS=2022-06-07T20:26:53.624296821Z Servers=[]

Component Ports

Make sure that all the ports needed are open under your firewall configuration.

Using UDMG with SELinux

Modify the file label so that NGINX (as a process labeled with the httpd_t context) can access the configuration file

# restorecon /etc/nginx/conf.d/*

Modify the file label so that NGINX (as a process labeled with the httpd_t context) can access the asset files

# semanage fcontext -a -t httpd_sys_content_t '/srv/www(/.*)?'
# restorecon -Rv /srv/www/

Allow NGINX to reverse proxy through the authentication proxy by setting the httpd_can_network_connect boolean

# setsebool -P httpd_can_network_connect 1

References

This document references the following documents.

Name	Location
Systemd	Understanding and Using Systemd - Linux.com
NGINX with SELinux	Modifying SELinux Settings for Full NGINX and NGINX Plus Functionality
PostgreSQL Client Authentication	21.1. The pg_hba.conf File
PostgreSQL Password Authentication	21.5. Password Authentication

Universal Data Mover Gateway 1.2

UDMG for Linux Installation

Installing and Configuring the Components

UDMG Admin UI

UDMG User setup

UDMG Server

UDMG Authentication Proxy

Configuration for LDAP Authentication

UDMG Agent Proxy

Agent Configuration

Client Configuration

Setup the Systemd Services

UDMG Server

UDMG Authentication Proxy

UDMG Agent Proxy

Agent Proxy Server Service

Agent Proxy Client Service

Component Ports

Using UDMG with SELinux

References