Pre-Installation / Upgrade Backups

The installation process overwrites the current files (exception: the configuration files are kept), effectively removing your modifications. Backing up these files will optimize the time it takes you to get up and running after installing or upgrading.

Note

If you are upgrading an installation of UDMG from any release prior to 1.3.0.0, you must uninstall the older version before installing the new version.
The binaries, services, configuration parameters, and environment variables have changed and this does not allow for a standard upgrade.

Category	Prior releases	Release 1.3
UDMG Admin UI assets	C:\UDMG\nginx\mft	C:\UDMG\nginx\udmg
Environment variables	WAARP_GATEWAY_ADDRESS MFT_AUTH_PROXY_CONFIG MFT_AGENT_PROXY_CONFIG	UDMG_SERVER_ADDRESS UDMG_AUTH_PROXY_CONFIG UDMG_AGENT_PROXY_CONFIG

It may be required to modify the work and data directories ownership or access rights and to update UDMG Server transfer rules to use new paths.

The configuration files must be reviewed and compared between the old and new versions.

Special attention is required for the AESpassphrase parameter for UDMG Server. It must be the path for the file that was used by the previous release and must be accessible by the new service user. It is recommended to set an absolute path in the configuration file.

Upgrading UDMG Server

The UDMG release version is stored in the database to ensure the data structure is compatbile with the version of the UDMG components.

After upgrading the component binaries and before starting the UDMG server it is required to perform the release migration step.

The udmg-server "migrate" command handles the necessay database updates and the setting of the internal version.

C:\UDMG\UDMG Server>udmg-server migrate /?
Usage:
  udmg-server [OPTIONS] migrate [migrate-OPTIONS] [version]

Help Options:
  /?                Show this help message
  /h, /help         Show this help message

[migrate command options]
      /c, /config:  The configuration file to use
      /d, /dry-run  Simulate the migration but does not commit the changes
      /l, /list     List Migrations
      /f, /file:    Writes the migration commands into a file instead of sending them to the database
      /v, /verbose  Show verbose debug information. Can be repeated to increase verbosity

[migrate command arguments]
  version:           The version to which the database should be migrated

The configuration file is the one that will be used for the server mode, with the parameter for accessing the target database.

To get the list of supported target version, use the list parameter. The last version is the release version of the udmg-server.

C:\UDMG\UDMG Server>udmg-server migrate /c "C:\UDMG\UDMG Server\server.ini" /list 
0.7.1
0.7.1-sb.1
0.7.1-sb.2
0.7.1-sb.3
1.3.0

It is recommended to set the verbose parameter (3 times) to follow the progress.

C:\UDMG\UDMG Server>udmg-server migrate /c "C:\UDMG\UDMG Server\server.ini" /v /v /v 1.3.0.0
[INFO ] Migration: Starting upgrade migration...
[INFO ] Migration: Applying migration 'Fix the 'sb_generic_group_join' table foreign keys'
[INFO ] Migration: Applying migration 'Fix the 'sb_user_group_member' table foreign keys'
[INFO ] Migration: Applying migration 'Fix the 'sb_user_group_permission' table foreign keys'
[INFO ] Migration: Applying migration 'Bump database version to 0.7.1-sb.2'
[INFO ] Migration: Applying migration 'Add description under 'local_agents' and 'remote_accounts''
[INFO ] Migration: Applying migration 'Add a 'revoked' column to the 'crypto_credentials' table'
[INFO ] Migration: Applying migration 'Bump database version to 0.7.1-sb.3'
[INFO ] Migration: Applying migration 'Add a 'email' column to the 'users' table'
[INFO ] Migration: Applying migration 'Create the 'access_token' table'
[INFO ] Migration: Applying migration 'Bump database version to 1.3.0'

After the migration, the service for UDMG Server can be started.

Installing and Configuring the Components

UDMG Admin UI

Extract the zip file for UDMG Admin UI, under the directory that was created during the NGINX installation.

C:\UDMG\nginx\udmg> tar -x -f udmg_admin_ui-<version>.zip

Validate that the service is working properly with curl

C:\>curl.exe http://localhost:80 -I
HTTP/1.1 200 OK
Server: nginx/1.23.0
Date: Thu, 07 Jul 2022 17:53:09 GMT
Content-Type: text/html
Content-Length: 7788
Last-Modified: Fri, 01 Jul 2022 13:58:15 GMT
Connection: keep-alive
ETag: "62befd77-1e6c"
Accept-Ranges: bytes

or with the browser:

UDMG Server

Create a directory C:\UDMG\UDMG Server
Install the binaries as udmg-server.exe and udmg-client.exe
Create the configuration file C:\UDMG\UDMG Server\server.ini with the following parameters:

Note

The lines starting with a colon ';' or a hash '#' are comments, describing the option or showing the default value.

The parameters must be adapted to your environment, in particular:

global: GatewayHome
log: LogLevel, LogTo, LogPath
admin: Host, Port
database: Type, Address, Name, User, Password

[global]
; The name given to identify this UDMG Server instance. If the the database is shared between multiple servers, this name MUST be unique across these servers.
GatewayName = udmg

; Default OS permission for created files
; FilePermissions = 700

; Default OS permission for created directories
; DirPermissions = 750

[paths]
; The root directory of the UDMG Server. By default, it is the working directory of the process.
; GatewayHome = "C:\UDMG\UDMG Server"

; The directory for all incoming files.
; DefaultInDir = in

; The directory for all outgoing files.
; DefaultOutDir = out

; The directory for all running transfer files.
; DefaultTmpDir = tmp

[log]
; All messages with a severity above this level will be logged. Possible values are DEBUG, INFO, WARNING, ERROR and CRITICAL.
Level = DEBUG

; The path to the file where the logs must be written. Special values 'stdout' and 'syslog' log respectively to the standard output and to the syslog daemon
; LogTo = stdout

; If LogTo is set on 'syslog', the logs will be written to this facility.
; SyslogFacility = local0

; The directory for the log files of the local servers, partners, and transfers. No default, if not provided then the detailled log feature is disabled.; LogPath = "C:\UDMG\UDMG Server\logs"

[admin]
; The address used by the admin interface.
Host = 0.0.0.0

; The port used by the admin interface. If the port is 0, a free port will automatically be chosen.
Port = 18080

; Path of the TLS certificate for the admin interface.
; TLSCert =

; Path of the key of the TLS certificate.
; TLSKey =

; API rate limiter: number of allowed requests per client IP, per second. After that HTTP code 429 is returned. Disabled if 0 or not provided.
; RateLimit = 0

[database]
; Name of the RDBMS used for the UDMG Server database. Possible values: sqlite, mysql, postgresql
Type = postgresql

; Address of the database
Address = localhost

; The name of the database
Name = udmg

; The name of the database user
User = udmg_user

; The password of the database user
Password = udmg_password

; Path of the database TLS certificate file.
; TLSCert =

; Path of the key of the TLS certificate file.
; TLSKey =

; The path to the file containing the passphrase used to encrypt account passwords using AES
; AESPassphrase = passphrase.aes

[controller]
; The frequency at which the database will be probed for new transfers
; Delay = 5s

; The maximum number of concurrent incoming transfers allowed on the UDMG Server (0 = unlimited).
; MaxTransferIn = 0

; The maximum number of concurrent outgoing transfers allowed on the UDMG Server (0 = unlimited).
; MaxTransferOut = 0

[sftp]
; Set to true to allow legacy and weak cipher algorithms: 3des-cbc, aes128-cbc, arcfour, arcfour128, arcfour256
; AllowLegacyCiphers = false

Note

AESPassphrase file is generated on first run if it does not exist.

Make sure to set verify the file location during upgrade and to have a backup. Without the correct AESPassphrase file, the passwords, the keys and the certificates will not be usable.

UDMG Authentication Proxy

Create a directory C:\UDMG\UDMG Auth Proxy
Install the binaries as udmg-auth-proxy.exe
Create the configuration file C:\UDMG\UDMG Auth Proxy\config.toml with the following parameters:

# Proxy Configuration
[proxy]
# Port, default "5000"
port = "5000"
# Network interface, default "0.0.0.0"
inet = "127.0.0.1"
# Enable recover on panic, default true, should be true for production environment
recover = true
# Enable Cross-Origin Resource Sharing (CORS), should be true for production environment
cors = true # CORS: List of origins that may access the resource. Optional. Default value "*" # domain = "*"
# Enable Request Track ID, default true
tracker = true
# Enable Request Logger, default true
logger = true
# Rate Limit IP Request over 1 second, default 0 (unlimited)
limit = 0
# Enable the Prometheus Metric Endpoint '/metric', default false
metrics = false # Enable CSRF token csrf = false# Service 'local' with direct authentication on the UDMG Server
[service.local]
# UDMG Server Listen Protocol
protocol = "http"[[service.local.targets]]
# UDMG Server Hostname or IP
hostname = "localhost"
# UDMG Server Port
port = 18080

Please refer to Authentication Methods for the LDAP and SSO authentication options.

UDMG Agent Proxy

Create a directory C:\UDMG\UDMG Agent
Install the binaries as udmg-agent-proxy-client.exe and udmg-agent-proxy-server.exe

Agent Proxy Server Configuration

Generate a SSH Key for the service.
For example with the ssh-keygen tool that is provided by Windows OpenSSH tools (Key-based authentication in OpenSSH for Windows), or by 3PP packages like Copssh - OpenSSH for Windows or Portable Git for Windows

# ssh-keygen -t rsa -q -N "" -f "C:\UDMG\UDMG Agent\agent_key"
# ssh-keygen -t rsa -q -N "" -f "C:\UDMG\UDMG Agent\client_key"

If OpenSSH is not installed or not available, the PuTTY tool can be used instead.

Use PuTTYgen to generate a key pair for the agent, more detailled instructions can be found here: Using public keys for SSH authentication

After generating the key, export it with OpenSSH format:

Create a configuration file as C:\UDMG\UDMG Agent\agent\agent.toml

[agent]
# Listen IP Address
hostname = "0.0.0.0"
# Listen Port
port = "2222"
# SSH Priv Key
ssh_key = "C:\UDMG\UDMG Agent\agent_key"
# SSH Public Key
ssh_key_pub = "C:\UDMG\UDMG Agent\agent_key.pub"

# Service User
username = "mft"
# Service Password
password = "61ee8b5601a84d5154387578466c8998848ba089"

The password key will be used for the client authentication.

Agent Proxy Client Configuration

Create a configuration file as C:\UDMG\UDMG Agent\client\client.toml

[client]
# UDMG Agent Proxy Hostname or IP
hostname = "localhost"
# UDMG Agent Proxy Listen Port
port = "2222"

# UDMG Agent Service User and password
username = "mft"
password = "61ee8b5601a84d5154387578466c8998848ba089"

# Default TTL to Connection Retry
ttl="5s"

# UDMG Agent Client Admin API
[client.api]
# Listen Port
port="2280"

# UDMG Server
[gateway]
# UDMG Server Hostname or IP
hostname = "localhost"
# UDMG Server Port
port = "18080"
# UDMG Server Username/Password
username = "admin"
password = "admin_password"

The password key will be used for the client authentication.

Setup the Windows Services

The UDMG components can be installed as Windows service with the WinSw tool.

NGINX Server

Download WinSW and copyy winsw.exe under C:\UDMG\nginx with the following name: nginx-service.exe
Create a new service definition file: nginx-service.yml

id: NGINX
name: NGINX Service
description: Runs the nginx.exe as a service
executable: C:\UDMG\nginx\nginx.exe
stopexecutable: C:\UDMG\nginx\nginx-stop.cmd
logpath: C:\UDMG\nginx\logs
logmode: roll

Create a stop script: nginx-stop.cmd

@echo off

cd /D %~dp0

if not exist logs\nginx.pid GOTO skip
nginx.exe -s quit
del logs\nginx.pid

:skip

taskkill /f /IM nginx.exe

Start the service and check the status:

C:\UDMG\nginx>nginx-service.exe install
2022-10-07 10:20:20,724 INFO - Installing service 'NGINX Service (NGINX)'...
2022-10-07 10:20:20,777 INFO - Service 'NGINX Service (NGINX)' was installed successfully.

C:\UDMG\nginx>nginx-service.exe status
Stopped

C:\UDMG\nginx>nginx-service.exe start
2022-10-07 10:20:40,213 INFO - Starting service 'NGINX Service (NGINX)'...
2022-10-07 10:20:41,327 INFO - Service 'NGINX Service (NGINX)' started successfully.

C:\UDMG\nginx>nginx-service.exe status
Started

UDMG Server

Download WinSW and copy winsw.exe under C:\UDMG\UDMG Server with the following name: udmg-server-service.exe
Create a new service definition file: udmg-server-service.yml

Here is the minimal version of the file with the required configuration parameters, make sure that the paths are correct

id: UDMG Servername: UDMG Serverdescription: UDMG Serverexecutable: C:\UDMG\UDMG Server\udmg-server.exestartArguments: server -c "C:\UDMG\UDMG Server\server.ini"workingdirectory: C:\UDMG\UDMG Serverpriority: Normal

Here is another version with the full set of parameters and their description, it can be used as a template for tweaking the service settings.

## UDMG server service configuration
##
## references:
## https://github.com/winsw/winsw/blob/v2.11.0/doc/yamlConfigFile.md
## https://github.com/winsw/winsw/blob/v2.11.0/examples/sample-allOption.yml

id: UDMG Server
name: UDMG Server
description: UDMG Server

## SECTION: Installation
## These options are being used during the installation only.
## Their modification will not take affect without the service re-installation.
##
## service account can be given during the installation with /p parameter
## waarp-gatewayd-service-user.exe install /p
## and then providing username
onFailure:
-
action: restart
delay: 10 sec
-
action: restart
delay: 60 sec
-
action: none
#resetFailure: 01:00:00

## SECTION: Executable management
executable: C:\UDMG\UDMG Server\udmg-server.exe
startArguments: >
server
-c "C:\UDMG\UDMG Server\server.ini"
workingdirectory: C:\UDMG\UDMG Server
priority: Normal

#stopTimeout: 15 sec
#stopParentProcessFirst: true

## SECTION: Service management
#startMode: Automatic
#delayedAutoStart: true

## SECTION:Logging
## Note: disabled due to WinSW bug https://github.com/winsw/winsw/issues/1016
#log:
# mode: roll-by-size-time
# sizeThreshold: 10240
# keepFiles: 8
# pattern: yyyyMMdd
# autoRollAtTime: 00:00:00

## SECTION: Environment setup

## SECTION: Extensions
## This configuration section allows specifying custom extensions.
## More info is available here: https://github.com/kohsuke/winsw/blob/v2.11.0/doc/extensions/extensions.md

Note

The rollover of the service log file is disabled as it causes a bug of the WinSW application, see https://github.com/winsw/winsw/issues/1016.

Start the service and check the status:

C:\UDMG\UDMG Server>udmg-server-service.exe install2022-10-07 10:07:48,385 INFO - Installing service 'UDMG Server (UDMG Server)'...2022-10-07 10:07:48,449 INFO - Service 'UDMG Server (UDMG Server)' was installed successfully.C:\UDMG\UDMG Server>udmg-server-service.exe statusStoppedC:\UDMG\UDMG Server>udmg-server-service.exe start2022-10-07 10:08:19,297 INFO - Starting service 'UDMG Server (UDMG Server)'...2022-10-07 10:08:20,418 INFO - Service 'UDMG Server (UDMG Server)' started successfully.C:\UDMG\UDMG Server>udmg-server-service.exe statusStarted

Be sure that the listen port and network interface is reachable by UDMG Authentication Proxy and UDMG Agent Client.

UDMG Authentication Proxy

Download WinSW and copy winsw.exe under C:\UDMG\UDMG Auth Proxy with the following name: udmg-auth-proxy-service.exe
Create a new service definition file: udmg-auth-proxy-service.yml

id: UDMG Auth Proxyname: UDMG Auth Proxydescription: UDMG Auth Proxyexecutable: C:\UDMG\UDMG Auth Proxy\udmg-auth-proxy.exepriority: Normalworkingdirectory: C:\UDMG\UDMG Auth Proxyenv: - name: UDMG_AUTH_PROXY_CONFIG value: 'C:\UDMG\UDMG Auth Proxy\config.toml'

Start the service and check the status:

C:\UDMG\UDMG Auth Proxy>udmg-auth-proxy-service.exe install2022-10-07 10:09:45,618 INFO - Installing service 'UDMG Auth Proxy (UDMG Auth Proxy)'...2022-10-07 10:09:45,683 INFO - Service 'UDMG Auth Proxy (UDMG Auth Proxy)' was installed successfully.C:\UDMG\UDMG Auth Proxy>udmg-auth-proxy-service.exe start2022-10-07 10:09:52,852 INFO - Starting service 'UDMG Auth Proxy (UDMG Auth Proxy)'...2022-10-07 10:09:54,053 INFO - Service 'UDMG Auth Proxy (UDMG Auth Proxy)' started successfully.C:\UDMG\UDMG Auth Proxy>udmg-auth-proxy-service.exe statusStarted

Be sure that the listen port and network interface is reachable by NGINX Server.

UDMG Agent Proxy

Agent Proxy Server Service

Download WinSW and copy winsw.exe under C:\UDMG\UDMG Agent\agent with the following name: udmg-agent-proxy-server-service.exe
Create a new service definition file: udmg-agent-proxy-server-service.yml

id: UDMG Agent Server
name: UDMG Agent Server
description: UDMG Agent Server
executable: C:\UDMG\UDMG Agent\agent\udmg-agent-proxy-server.exe
priority: Normal
workingdirectory: C:\UDMG\UDMG Agent\agent
env:
- name: MFT_AGENT_PROXY_CONFIG
value: 'C:\UDMG\UDMG Agent\agent\agent.toml'

Start the service and check the status:

C:\UDMG\UDMG Agent\agent>udmg-agent-proxy-server-service.exe install2022-10-07 10:09:45,618 INFO - Installing service 'UDMG Agent Server (UDMG Agent Server)'...2022-10-07 10:09:45,683 INFO - Service 'UDMG Agent Server (UDMG Agent Server)' was installed successfully. C:\UDMG\UDMG Agent\agent>udmg-agent-proxy-server-service.exe start2022-10-07 10:09:52,852 INFO - Starting service 'UDMG Agent Server (UDMG Agent Server)'...2022-10-07 10:09:54,053 INFO - Service 'UDMG Agent Server (UDMG Agent Server)' started successfully. C:\UDMG\UDMG Agent\agent>udmg-agent-proxy-server-service.exe statusStarted

Be sure that the listen port and network interface is reachable by UDMG Agent Client .

Agent Proxy Client Service

Download WinSW and copy winsw.exe under C:\UDMG\UDMG Agent\client with the following name: udmg-agent-proxy-client-service.exe
Create a new service definition file: udmg-agent-proxy-client-service.yml

id: UDMG Agent Client
name: UDMG Agent Client
description: UDMG Agent Client
executable: C:\UDMG\UDMG Agent\client\udmg-agent-proxy-client.exe
priority: Normal
workingdirectory: C:\UDMG\UDMG Agent\client
env:
- name: MFT_AGENT_PROXY_CONFIG
value: 'C:\UDMG\UDMG Agent\client\agent.toml'

Start the service and check the status:

C:\UDMG\UDMG Agent\client>udmg-agent-proxy-client-service.exe install2022-10-07 10:09:45,618 INFO - Installing service 'UDMG Agent Client(UDMG Agent Client)'...2022-10-07 10:09:45,683 INFO - Service 'UDMG Agent Client (UDMG Agent Client)' was installed successfully.C:\UDMG\UDMG Agent\client>udmg-agent-proxy-client-service.exe start2022-10-07 10:09:52,852 INFO - Starting service 'UDMG Agent Client (UDMG Agent Client)'...2022-10-07 10:09:54,053 INFO - Service 'UDMG Agent Client(UDMG Agent Client)' started successfully.C:\UDMG\UDMG Agent\client>udmg-agent-proxy-client-service.exe statusStarted

Ports Configuration

See Network Requirements

References

This document references the following documents.

Name	Location
PostgreSQL Client Authentication	21.1. The pg_hba.conf File
PostgreSQL Password Authentication	21.5. Password Authentication
Guide on setting up Nginx as a service on Windows	https://github.com/sheggi/win-service-nginx

Universal Data Mover Gateway 1.3

UDMG for Windows Installation

Pre-Installation / Upgrade Backups

Upgrading UDMG Server

Installing and Configuring the Components

UDMG Admin UI

UDMG Server

UDMG Authentication Proxy

UDMG Agent Proxy

Agent Proxy Server Configuration

Agent Proxy Client Configuration

Setup the Windows Services

NGINX Server

UDMG Server

UDMG Authentication Proxy

UDMG Agent Proxy

Agent Proxy Server Service

Agent Proxy Client Service

Ports Configuration

References