Installing and Configuring the Components

UDMG Admin UI

Uncompress the tar file for MFT Admin UI, under the directory that we created during the configuration.

C:\UDMG\nginx\mft> tar -x -f mft_admin_ui-<version>.zip

Validate that the service is working properly with curl

C:\>curl.exe http://localhost:8080 -I
HTTP/1.1 200 OK
Server: nginx/1.23.0
Date: Thu, 07 Jul 2022 17:53:09 GMT
Content-Type: text/html
Content-Length: 7788
Last-Modified: Fri, 01 Jul 2022 13:58:15 GMT
Connection: keep-alive
ETag: "62befd77-1e6c"
Accept-Ranges: bytes

or with the browser:

UDMG Server

Create a directory C:\UDMG\UDMG Gateway
Install the binaries as waarp-gatewayd.exe and waarp-gateway.exe
Create the configuration file C:\UDMG\UDMG Gateway\server.ini with the following parameters:

[global]
; The name given to identify this gateway instance. If the the database is shared between multiple gateways, this name MUST be unique across these gateways.
GatewayName = mft_waarp_gateway

; Default OS permission for created files
; FilePermissions = 700

; Default OS permission for created directories
; DirPermissions = 750

[paths]
; The root directory of the gateway. By default, it is the working directory of the process.
; GatewayHome = /home/mft

; The directory for all incoming files.
; DefaultInDir = in

; The directory for all outgoing files.
; DefaultOutDir = out

; The directory for all running transfer files.
; DefaultTmpDir = tmp

[log]
; All messages with a severity above this level will be logged. Possible values are DEBUG, INFO, WARNING, ERROR and CRITICAL.
Level = DEBUG

; The path to the file where the logs must be written. Special values 'stdout' and 'syslog' log respectively to the standard output and to the syslog daemon
; LogTo = stdout

; If LogTo is set on 'syslog', the logs will be written to this facility.
; SyslogFacility = local0

[admin]
; The address used by the admin interface.
Host = 0.0.0.0

; The port used by the admin interface. If the port is 0, a free port will automatically be chosen.
Port = 18080

; Path of the TLS certificate for the admin interface.
; TLSCert =

; Path of the key of the TLS certificate.
; TLSKey =

[database]
; Name of the RDBMS used for the gateway database. Possible values: sqlite, mysql, postgresql
Type = postgresql

; Address of the database
Address = localhost

; The name of the database
Name = mft_waarp_gateway

; The name of the gateway database user
User = mft_waarp_gateway_user

; The password of the gateway database user
Password = mft_waarp_gateway_password

; Path of the database TLS certificate file.
; TLSCert =

; Path of the key of the TLS certificate file.
; TLSKey =

; The path to the file containing the passphrase used to encrypt account passwords using AES
; AESPassphrase = passphrase.aes

[controller]
; The frequency at which the database will be probed for new transfers
Delay = 5s

; The maximum number of concurrent incoming transfers allowed on the gateway (0 = unlimited).
; MaxTransferIn = 0

; The maximum number of concurrent outgoing transfers allowed on the gateway (0 = unlimited).
; MaxTransferOut = 0

[sftp]
; Set to true to allow legacy and weak cipher algorithms: 3des-cbc, aes128-cbc, arcfour, arcfour128, arcfour256
; AllowLegacyCiphers = false

UDMG Authentication Proxy

Create a directory C:\UDMG\MFT Auth Proxy
Install the binaries as mft_auth_proxy_server.exe
Create the configuration file C:\UDMG\MFT Auth Proxy\config.toml with the following parameters:

# Proxy Configuration
[proxy]
# Port, default "5000"
port = "5000"
# Network interface, default "0.0.0.0"
inet = "127.0.0.1"
# Enable recover on panic, default true, should be true for production environment
recover = true
# Enable Cross-Origin Resource Sharing (CORS), should be true for production environment
cors = true
# Enable Request Track ID, default true
tracker = true
# Enable Request Logguer, default true
logger = true
# Rate Limit IP Request over 1 second, default 0 (unlimited)
limit = 0
# Enable the Prometheus Metric Endpoint '/metric', default false
metrics = false

# Service 'local' with direct authentication on the waarp gateway
[service.local]
# MFT Waarp Gateway Listen Protocol
protocol = "http"

[[service.local.targets]]
# MFT Waarp Gateway Hostname or IP
hostname = "localhost"
# MFT Waarp Gateway Port
port = 18080

# Service 'mft' with direct authentication on the waarp gateway
[service.mft]
# MFT Waarp Gateway Listen Protocol
protocol = "http"

[[service.mft.targets]]
# MFT Waarp Gateway Hostname or IP
hostname = "localhost"
# MFT Waarp Gateway Port
port = 18080

Configuration for LDAP Authentication

The UDMG Authentication Proxy is capable to use a LDAP Service to authenticate users for UDMG Admin UI:

# Proxy Configuration
[proxy]
# Port, default "5000"
port = "5000"
# Network interface, default "0.0.0.0"
inet = "127.0.0.1"
# Enable recover on panic, default true, should be true for production environment
recover = true
# Enable Cross-Origin Resource Sharing (CORS), should be true for production environment
cors = true
# Enable Request Track ID, default true
tracker = true
# Enable Request Logguer, default true
logger = true
# Rate Limit IP Request over 1 second, default 0 (unlimited)
limit = 0
# Enable the Prometheus Metric Endpoint '/metric', default false
metrics = false

# Service 'local' with direct authentication on the waarp gateway
[service.local]
# MFT Waarp Gateway Listen Protocol
protocol = "http"

[[service.local.targets]]
# MFT Waarp Gateway Hostname or IP
hostname = "localhost"
# MFT Waarp Gateway Port
port = 18080

# Service 'mft' with LDAP Authentication
[service.mft]
# MFT Waarp Gateway Listen Protocol
protocol = "http"

# This option is breaking glass option for admins, they will not reach the LDAP service during the Auth Stage admins = ["admin"]

[[service.mft.targets]]
# MFT Waarp Gateway Hostname or IP
hostname = "localhost"
# MFT Waarp Gateway Port
port = 18080

# User for Sync Password Between Ldap and Proxy
[service.mft.credential]
# Pre - Setup user under MFT Waarp Gateway
username = "ldap_sync"
password = "ldap_password"

# LDAP Configuration
[service.mft.auth.ldap]
# LDAP Server DC with OU
dn = "ou=users,dc=stonebranch,dc=com"
# LDAP Server FQDN or IP
hostname = "myldap.server.fqdn.com"
# LDAP Server pORT
port = 1389

The LDAP replication requires a user with permission for creating and updating users. For example to create the 'ldap_sync' user with the command line interface:

waarp_gateway user add -u ldap_sync -p ldap_password -r 'U=rw'
In case of successful authentication on the LDAP, the user is created with default read permission in the internal UDMG database if it does not exist. Otherwise the credentials are updated in the database to allow for authentication on the REST and CLI interfaces.

UDMG Agent Proxy

Create a directory C:\UDMG\UDMG Agent
Install the binaries as mft_agent_proxy_client.exe and mft_agent_proxy_server.exe

Agent Configuration

Generate a SSH Key for the service.
For example with the ssh-keygen tool that is provided by Windows OpenSSH tools (Key-based authentication in OpenSSH for Windows), or by 3PP packages like Copssh - OpenSSH for Windows or Portable Git for Windows

# ssh-keygen -t rsa -q -N "" -f "C:\UDMG\UDMG Agent\agent_key"
# ssh-keygen -t rsa -q -N "" -f "C:\UDMG\UDMG Agent\client_key"

If OpenSSH is not installed or not available, the PuTTY tool can be used instead.

Use PuTTYgen to generate a key pair for the agent, more detailled instructions can be found here: Using public keys for SSH authentication

After generating the key, export it with OpenSSH format:

Create a configuration file as C:\UDMG\UDMG Agent\agent\agent.toml

[agent]
# Listen IP Address
hostname = "0.0.0.0"
# Listen Port
port = "2222"
# SSH Priv Key
ssh_key = "C:\UDMG\UDMG Agent\agent_key"
# SSH Public Key
ssh_key_pub = "C:\UDMG\UDMG Agent\agent_key.pub"

# Service User
username = "mft"
# Service Password
password = "61ee8b5601a84d5154387578466c8998848ba089"

The password key will be used for the client authentication.

Client Configuration

Create a configuration file as C:\UDMG\UDMG Agent\client\client.toml

[client]
# UDMG Agent Proxy Hostname or IP
hostname = "localhost"
# UDMG Agent Proxy Listen Port
port = "2222"

# SSH Priv Key
ssh_key = "C:\UDMG\UDMG Agent\client_key"
# SSH Public Key
ssh_key_pub = "C:\UDMG\UDMG Agent\client_key.pub"

# Service User
username = "mft"
# Service Password
password = "61ee8b5601a84d5154387578466c8998848ba089"

# Default TTL to Connection Retry
ttl="5s"

# UDMG Agent Client Admin API
[client.api]
# Listen Port
port="2280"

# MFT Waarp Gateway
[gateway]
# MFT Waarp Gateway Hostname or IP
hostname = "localhost"
# MFT Waarp Gateway Port
port = "18080"
# MFT Waarp Gateway Username/Password
username = "admin"
password = "admin_password"

The password key will be used for the client authentication.

Setup the Windows Services

The UDMG components can be installed as Windows service with the WinSW tool.

NGINX Server

Download WinSW and copy winsw.exe under C:\UDMG\nginx with the following name: nginx-service.exe
Create a new service definition file: nginx-service.yml

id: NGINX
name: NGINX Service
description: Runs the nginx.exe as a service
executable: C:\UDMG\nginx\nginx.exe
stopexecutable: C:\UDMG\nginx\nginx-stop.cmd
logpath: C:\UDMG\nginx\logs
logmode: roll

Create a stop script: nginx-stop.cmd

@echo off

cd /D %~dp0

if not exist logs\nginx.pid GOTO skip
nginx.exe -s quit
del logs\nginx.pid

:skip

taskkill /f /IM nginx.exe

Start the service and check the status:

C:\UDMG\nginx>nginx-service.exe install
2022-10-07 10:20:20,724 INFO - Installing service 'NGINX Service (NGINX)'...
2022-10-07 10:20:20,777 INFO - Service 'NGINX Service (NGINX)' was installed successfully.

C:\UDMG\nginx>nginx-service.exe status
Stopped

C:\UDMG\nginx>nginx-service.exe start
2022-10-07 10:20:40,213 INFO - Starting service 'NGINX Service (NGINX)'...
2022-10-07 10:20:41,327 INFO - Service 'NGINX Service (NGINX)' started successfully.

C:\UDMG\nginx>nginx-service.exe status
Started

UDMG Server

Download WinSW and copy winsw.exe under C:\UDMG\UDMG Gateway with the following name: waarp-gatewayd-service.exe
Create a new service definition file: waarp-gatewayd-service.yml

Here is the minimal version of the file with the required configuration parameters, make sure that the paths are correct

id: UDMG Servername: UDMG Serverdescription: UDMG Serverexecutable: C:\UDMG\UDMG Server\udmg-server.exestartArguments: server -c "C:\UDMG\UDMG Server\server.ini"workingdirectory: C:\UDMG\UDMG Serverpriority: Normal

Here is another version with the full set of parameters and their description, it can be used as a template for tweaking the service settings.

## UDMG server service configuration
##
## references:
## https://github.com/winsw/winsw/blob/v2.11.0/doc/yamlConfigFile.md
## https://github.com/winsw/winsw/blob/v2.11.0/examples/sample-allOption.yml

id: UDMG Server
name: UDMG Server
description: UDMG Server

## SECTION: Installation
## These options are being used during the installation only.
## Their modification will not take affect without the service re-installation.
##
## service account can be given during the installation with /p parameter
## waarp-gatewayd-service-user.exe install /p
## and then providing username
onFailure:
-
action: restart
delay: 10 sec
-
action: restart
delay: 60 sec
-
action: none
#resetFailure: 01:00:00

## SECTION: Executable management
executable: C:\UDMG\UDMG Server\udmg-server.exe
startArguments: >
server
-c "C:\UDMG\UDMG Server\server.ini"
workingdirectory: C:\UDMG\UDMG Server
priority: Normal

#stopTimeout: 15 sec
#stopParentProcessFirst: true

## SECTION: Service management
#startMode: Automatic
#delayedAutoStart: true

## SECTION:Logging
## Note: disabled due to WinSW bug https://github.com/winsw/winsw/issues/1016
#log:
# mode: roll-by-size-time
# sizeThreshold: 10240
# keepFiles: 8
# pattern: yyyyMMdd
# autoRollAtTime: 00:00:00

## SECTION: Environment setup

## SECTION: Extensions
## This configuration section allows specifying custom extensions.
## More info is available here: https://github.com/kohsuke/winsw/blob/v2.11.0/doc/extensions/extensions.md

Note

The rollover of the service log file is disabled as it causes a bug of the WinSW application, see https://github.com/winsw/winsw/issues/1016.

Start the service and check the status:

C:\UDMG\UDMG Gateway>waarp-gatewayd-service.exe install2022-10-07 10:07:48,385 INFO - Installing service 'UDMG Waarp Gateway Server (UDMG Server)'...2022-10-07 10:07:48,449 INFO - Service 'UDMG Waarp Gateway Server (UDMG Server)' was installed successfully.C:\UDMG\UDMG Gateway>waarp-gatewayd-service.exe statusStoppedC:\UDMG\UDMG Gateway>waarp-gatewayd-service.exe start2022-10-07 10:08:19,297 INFO - Starting service 'UDMG Waarp Gateway Server (UDMG Server)'...2022-10-07 10:08:20,418 INFO - Service 'UDMG Waarp Gateway Server (UDMG Server)' started successfully.C:\UDMG\UDMG Gateway>waarp-gatewayd-service.exe statusStarted

Be sure that the listen port and network interface is reachable by UDMG Authentication Proxy and UDMG Agent Client.

UDMG Authentication Proxy

Download WinSW and copy winsw.exe under C:\UDMG\UDMG Auth Proxy with the following name: mft_auth_proxy_server-service.exe
Create a new service definition file: mft_auth_proxy_server-service.yml

id: UDMG Auth Proxyname: UDMG Auth Proxydescription: UDMG Auth Proxyexecutable: C:\UDMG\UDMG Auth Proxy\mft_auth_proxy_server.exepriority: Normalworkingdirectory: C:\UDMG\UDMG Auth Proxyenv: - name: MFT_AUTH_PROXY_CONFIG value: 'C:\UDMG\UDMG Auth Proxy\config.toml'

Start the service and check the status:

C:\UDMG\UDMG Auth Proxy>mft_auth_proxy_server-service.exe install2022-10-07 10:09:45,618 INFO - Installing service 'UDMG Auth Proxy (UDMG Auth Proxy)'...2022-10-07 10:09:45,683 INFO - Service 'UDMG Auth Proxy (UDMG Auth Proxy)' was installed successfully.C:\UDMG\UDMG Auth Proxy>mft_auth_proxy_server-service.exe start2022-10-07 10:09:52,852 INFO - Starting service 'UDMG Auth Proxy (UDMG Auth Proxy)'...2022-10-07 10:09:54,053 INFO - Service 'UDMG Auth Proxy (UDMG Auth Proxy)' started successfully.C:\UDMG\UDMG Auth Proxy>mft_auth_proxy_server-service.exe statusStarted

Be sure that the listen port and network interface is reachable by NGINX Server.

UDMG Agent Proxy

Agent Proxy Server Service

Download WinSW and copy winsw.exe under C:\UDMG\UDMG Agent\agent with the following name: mft_agent_proxy_server-service.exe
Create a new service definition file: mft_agent_proxy_server-service.yml

id: UDMG Agent Server
name: UDMG Agent Server
description: UDMG Agent Server
executable: C:\UDMG\UDMG Agent\agent\mft_agent_proxy_server.exe
priority: Normal
workingdirectory: C:\UDMG\UDMG Agent\agent
env:
- name: MFT_AGENT_PROXY_CONFIG
value: 'C:\UDMG\UDMG Agent\agent\agent.toml'

Start the service and check the status:

C:\UDMG\UDMG Agent\agent>mft_agent_proxy_server-service.exe install2022-10-07 10:09:45,618 INFO - Installing service 'UDMG Agent Server (UDMG Agent Server)'...2022-10-07 10:09:45,683 INFO - Service 'UDMG Agent Server (UDMG Agent Server)' was installed successfully. C:\UDMG\UDMG Agent\agent>mft_agent_proxy_server-service.exe start2022-10-07 10:09:52,852 INFO - Starting service 'UDMG Agent Server (UDMG Agent Server)'...2022-10-07 10:09:54,053 INFO - Service 'UDMG Agent Server (UDMG Agent Server)' started successfully. C:\UDMG\UDMG Agent\agent>mft_agent_proxy_server-service.exe statusStarted

Be sure that the listen port and network interface is reachable by UDMG Agent Client .

Agent Proxy Client Service

Copy winsw.exe under C:\UDMG\UDMG Agent\client with the following name: mft_agent_proxy_client-service.exe
Create a new service definition file: mft_agent_proxy_client-service.yml

id: UDMG Agent Client
name: UDMG Agent Client
description: UDMG Agent Client
executable: C:\UDMG\UDMG Agent\client\mft_agent_proxy_client.exe
priority: Normal
workingdirectory: C:\UDMG\UDMG Agent\client
env:
- name: MFT_AGENT_PROXY_CONFIG
value: 'C:\UDMG\UDMG Agent\client\agent.toml'

Start the service and check the status:

C:\UDMG\UDMG Agent\client>mft_agent_proxy_server-service.exe install2022-10-07 10:09:45,618 INFO - Installing service 'UDMG Agent Server (UDMG Agent Server)'...2022-10-07 10:09:45,683 INFO - Service 'UDMG Agent Server (UDMG Agent Server)' was installed successfully.C:\UDMG\UDMG Agent\client>mft_agent_proxy_server-service.exe start2022-10-07 10:09:52,852 INFO - Starting service 'UDMG Agent Server (UDMG Agent Server)'...2022-10-07 10:09:54,053 INFO - Service 'UDMG Agent Server (UDMG Agent Server)' started successfully.C:\UDMG\UDMG Agent\client>mft_agent_proxy_server-service.exe statusStarted

Component Ports

Make sure that all the ports needed are open under your firewall configuration.

References

This document references the following documents.

Name	Location
PostgreSQL Client Authentication	21.1. The pg_hba.conf File
PostgreSQL Password Authentication	21.5. Password Authentication
Guide on setting up Nginx as a service on Windows	https://github.com/sheggi/win-service-nginx

Universal Data Mover Gateway 1.2

UDMG for Windows Installation

Installing and Configuring the Components

UDMG Admin UI

UDMG Server

UDMG Authentication Proxy

Configuration for LDAP Authentication

UDMG Agent Proxy

Agent Configuration

Client Configuration

Setup the Windows Services

NGINX Server

UDMG Server

UDMG Authentication Proxy

UDMG Agent Proxy

Agent Proxy Server Service

Agent Proxy Client Service

Component Ports

References