Versions Compared

Old Version 2

changes.mady.by.user Olivier Michaut

Saved on

compared with

New Version Current

changes.mady.by.user Olivier Michaut

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Panel

Table of Contents

Pre-Installation / Upgrade Backups

The installation process overwrites the current files (exception: the configuration files are kept), effectively removing your modifications. Backing up these files will optimize the time it takes you to get up and running after installing or upgrading.

...

titleNote

If you are upgrading an installation of UDMG from any release prior to 1.3.0.0, you must uninstall the older version before installing the new version.
The binaries, services, configuration parameters, and environment variables have changed and this does not allow for a standard upgrade.

...

  • WAARP_GATEWAY_ADDRESS
  • MFT_AUTH_PROXY_CONFIG
  • MFT_AGENT_PROXY_CONFIG

...

  • UDMG_SERVER_ADDRESS
  • UDMG_AUTH_PROXY_CONFIG
  • UDMG_AGENT_PROXY_CONFIG

It may be required to modify the work and data directories ownership or access rights and to update UDMG Server transfer rules to use new paths.

The configuration files must be reviewed and compared between the old and new versions.

Special attention is required for the AESpassphrase parameter for UDMG Server. It must be the path for the file that was used by the previous release and must be accessible by the new service user. It is recommended to set an absolute path in the configuration file.

Upgrading UDMG Server

The UDMG release version is stored in the database to ensure the data structure is compatbile with the version of the UDMG components.

After upgrading the component binaries and before starting the UDMG server it is required to perform the release migration step.

The udmg-server "migrate" command handles the necessay database updates and the setting of the internal version.

Code Block
C:\UDMG\UDMG Server>udmg-server migrate /?
Usage:
  udmg-server [OPTIONS] migrate [migrate-OPTIONS] [version]

Help Options:
  /?                Show this help message
  /h, /help         Show this help message

[migrate command options]
      /c, /config:  The configuration file to use
      /d, /dry-run  Simulate the migration but does not commit the changes
      /l, /list     List Migrations
      /f, /file:    Writes the migration commands into a file instead of sending them to the database
      /v, /verbose  Show verbose debug information. Can be repeated to increase verbosity

[migrate command arguments]
  version:           The version to which the database should be migrated

The configuration file is the one that will be used for the server mode, with the parameter for accessing the target database.

To get the list of supported target version, use the list parameter. The last version is the release version of the udmg-server.

Code Block
C:\UDMG\UDMG Server>udmg-server migrate /c "C:\UDMG\UDMG Server\server.ini" /list 
0.7.1
0.7.1-sb.1
0.7.1-sb.2
0.7.1-sb.3
1.3.0
1.4.0

It is recommended to set the verbose parameter (3 times) to follow the progress.

Code Block
C:\UDMG\UDMG Server>udmg-server migrate /c "C:\UDMG\UDMG Server\server.ini" /v /v /v 1.4.0.0
[INFO ] Migration: Starting upgrade migration...
[INFO ] Migration: Applying migration 'Bump database version to 1.4.0'

...


Panel

Table of Contents

For the UDMG Web Transfer Client, please refer to UDMG Web Transfer Client for Windows Installation.

Upgrading Universal Data Mover Gateway

Upgrading UDMG refers to the increase of a currently installed pre-1.4.x Version, Release, or Modification level of UDMG (1.3.x, 1.2.x, 1.1.x, 1.0.x) to UDMG 1.4.x.

Note
titleNote

If you are upgrading an installation of UDMG from any release prior to 1.3.0.0, you must uninstall the older version before installing the new version.
The binaries, services, configuration parameters, and environment variables have changed and this does not allow for a standard upgrade.


CategoryPrior releasesRelease 1.3
UDMG Admin UI assetsC:\UDMG\nginx\mftC:\UDMG\nginx\udmg
Environment variables
  • WAARP_GATEWAY_ADDRESS
  • MFT_AUTH_PROXY_CONFIG
  • MFT_AGENT_PROXY_CONFIG
  • UDMG_SERVER_ADDRESS
  • UDMG_AUTH_PROXY_CONFIG
  • UDMG_AGENT_CONFIG

It may be required to modify the work and data directories ownership or access rights and to update UDMG Server transfer rules to use new paths.

The configuration files must be reviewed and compared between the old and new versions.

Special attention is required for the AESpassphrase parameter for UDMG Server. It must be the path for the file that was used by the previous release and must be accessible by the new service user. It is recommended to set an absolute path in the configuration file.

Anchor
backup
backup
Pre-Installation / Upgrade Backups

Make sure to have a backup of the configuration files as it will optimize the time it takes you to get up and running after upgrading.


C:\UDMG\UDMG Server\server.ini
C:\UDMG\UDMG Auth Proxy\config.toml
C:\UDMG\UDMG Agent\agent\agent.toml
C:\UDMG\UDMG Agent\client\client.toml

Anchor
db_migration
db_migration
Release migration

The UDMG release version is stored in the database to ensure the data structure is compatible with the version of the UDMG components.

After upgrading the component binaries and before starting the UDMG server it is required to perform the release migration step.


The udmg-server "migrate" command handles the necessary database updates and the setting of the internal version.

Code Block
C:\UDMG\UDMG Server>udmg-server migrate /?
Usage:
  udmg-server [OPTIONS] migrate [migrate-OPTIONS] [version]

Help Options:
  /?                Show this help message
  /h, /help         Show this help message

[migrate command options]
      /c, /config:  The configuration file to use
      /d, /dry-run  Simulate the migration but does not commit the changes
      /l, /list     List Migrations
      /f, /file:    Writes the migration commands into a file instead of sending them to the database
      /v, /verbose  Show verbose debug information. Can be repeated to increase verbosity

[migrate command arguments]
  version:           The version to which the database should be migrated

The configuration file is the one that will be used for the server mode, with the parameter for accessing the target database.

To get the list of supported target version, use the list parameter. The last version is the release version of the udmg-server.

Code Block
C:\UDMG\UDMG Server>udmg-server migrate /c "C:\UDMG\UDMG Server\server.ini" /list 
0.7.1
0.7.1-sb.1
0.7.1-sb.2
0.7.1-sb.3
1.3.0
1.4.0
1.4.1

It is recommended to set the verbose parameter (3 times) to follow the progress.

Code Block
C:\UDMG\UDMG Server>udmg-server migrate /c "C:\UDMG\UDMG Server\server.ini" /v /v /v 1.4.1
[INFO ] Migration: Starting upgrade migration...
[INFO ] Migration: Applying migration 'Bump database version to 1.4.1'

After the migration, the service for UDMG Server can be started.

Upgrading Universal Data Mover Gateway for Windows

Step 1

Contact your Stonebranch representative or the Customer Support to receive the software binaries.

Step 2

Perform the recommended 216891684 of configuration files.

Step 3

Stop the components services.

The exact steps depend on the system architecture and the deployed components, for example:


C:\UDMG\nginx>nginx-service.exe stop
C:\UDMG\UDMG Auth Proxy\udmg-auth-proxy-service.exe stop
C:\UDMG\UDMG Server\udmg-server-service.exe stop
C:\UDMG\UDMG Agent\client\udmg-agent-client-service.exe stop
C:\UDMG\UDMG Agent\agent\udmg-agent-server-service.exe stop

Step 4

Upgrade the UDMG component binaries by copying the release files to their target location:

copy udmg-server.exe "C:\UDMG\UDMG Server\udmg-server.exe"
copy udmg-client.exe "C:\UDMG\UDMG Server\udmg-client.exe"
copy icap-client.exe "C:\UDMG\UDMG Server\icap-client.exe"
copy udmg-auth-proxy.exe "C:\UDMG\UDMG Auth Proxy\udmg-auth-proxy.exe"
copy udmg-agent-client.exe "C:\UDMG\UDMG Agent\client\udmg-agent-client.exe"
copy udmg-agent-server.exe "C:\UDMG\UDMG Agent\agent\udmg-agent-server.exe" 


Upgrade the UDMG Admin UI:

C:\UDMG\nginx> move udmg udmg_BACKUP

C:\UDMG\nginx> cd udmg

C:\UDMG\nginx\udmg> tar -x -f "udmg-admin-ui-1.4.1.0 build.7.zip"

Step 5

Review the component configuration files.
Refer to each component installation section below for the list of parameters.
C:\UDMG\UDMG Server\server.ini
C:\UDMG\UDMG Auth Proxy\config.toml
C:\UDMG\UDMG Agent\agent\agent.toml
C:\UDMG\UDMG Agent\client\client.toml

Step 6Perform the release migration.
Step 7

Start the components services.

The exact steps depend on the system architecture and the deployed components, for example:

C:\UDMG\UDMG Server\udmg-server-service.exe start
C:\UDMG\UDMG Auth Proxy\\udmg-auth-proxy-service.exe start
C:\UDMG\nginx>nginx-service.exe start
C:\UDMG\UDMG Agent\client\udmg-agent-client-service.exe start
C:\UDMG\UDMG Agent\agent\udmg-agent-server-service.exe start

Installing and Configuring the Components

...

Panel

C:\UDMG\nginx\udmg> tar -x -f udmg_admin_ui-<version>.zip-admin-ui-<version>.zip

  • The zip file can now be deleted.
  • Validate that the service is working properly with the 'curl' command:
Panel

C:\>curl.exe http://localhost:80 -I
HTTP/1.1 200 OK
Server: nginx/1.23.0
Date: Thu, 07 Jul 2022 17:53:09 GMT
Content-Type: text/html
Content-Length: 7788
Last-Modified: Fri, 01 Jul 2022 13:58:15 GMT
Connection: keep-alive
ETag: "62befd77-1e6c"
Accept-Ranges: bytes

...

  • Create a directory C:\UDMG\UDMG Server

  • Install Copy the binaries as udmg-server.exe and udmg-client.exe

  • Create the configuration file C:\UDMG\UDMG Server\server.ini with the following parameters:
Note
titleNote

The lines starting with a colon ';' or a hash '#' are comments, describing the option or showing the default value.

The parameters must be adapted to your environment, in particular:

  • global section: GatewayHome
  • log section: LogLevel, LogTo, LogPath
  • admin section: Host, Port
  • database section: Type, Address, Name, User, Password

...

Panel

[global]
; The name given to identify this UDMG Server instance. If the the database is shared between multiple servers, this name MUST be unique across these servers.
GatewayName = udmg

; Default OS permission for created files
; FilePermissions = 700770

; Default OS permission for created directories
; DirPermissions = 750770


[paths]
; The root directory of the UDMG Server. By default, it is the working directory of the process.
; GatewayHome = "C:\UDMG\UDMG Server"

; The directory for all incoming files.
; DefaultInDir = in

; The directory for all outgoing files.
; DefaultOutDir = out

; The directory for all running transfer files.
; DefaultTmpDir = tmp


[log]
; All messages with a severity above this level will be logged. Possible values are TRACE, DEBUG, INFO, WARNING, ERROR and CRITICAL.
Level = DEBUG

; The path to the file where the logs must be written. Special values 'stdout' and 'syslog' log respectively to the standard output and to the syslog daemon
; LogTo = stdout

; If LogTo is set on 'syslog', the logs will be written to this facility.
; SyslogFacility = local0

; The directory for the log files of the local servers, partners, and transfers. No default, if not provided then the detailled detailed log feature is disabled.
; LogPath = "C:\UDMG\UDMG Server\logs"


[admin]
; The address used by the admin interface.
Host = 0.0.0.0

; The port used by the admin interface. If the port is 0, a free port will automatically be chosen.
Port = 18080

; Path of the TLS certificate for the admin interface.
; TLSCert =

; Path of the key of the TLS certificate.
; TLSKey =

;

API

rate

limiter:

number

of

allowed

requests

per

client

IP,

per

second.

After

that

HTTP

code

429

is

returned.

Disabled

if

0

or

not

provided.

;

RateLimit

=

0



[database]
; Name of the RDBMS used for the UDMG Server database. Possible values: sqlite, mysql, postgresql, oracle, mssql
Type = postgresql

; Address of the database
Address = localhost

; The name of the database
Name = udmg

; The name of the database user
User = udmg_user

; The password of the database user
Password = udmg_password

; Path of the database TLS certificate file.
; TLSCert =

; Path of the key of the TLS certificate file.
; TLSKey =

; The path to the file containing the passphrase used to encrypt account passwords using AES
; AESPassphrase = passphrase.aes

[controller]
; The frequency at which the database will be probed for new transfers
; Delay = 5s

; The maximum number of concurrent incoming transfers allowed on the UDMG Server (0 = unlimited).
; MaxTransferIn = 0

; The maximum number of concurrent outgoing transfers allowed on the UDMG Server (0 = unlimited).
; MaxTransferOut = 0


[sftp]
; Set to true to allow legacy and weak cipher algorithms: 3des-cbc, aes128-cbc, arcfour, arcfour128, arcfour256
; AllowLegacyCiphers = false

...

  • Create a directory C:\UDMG\UDMG Auth Proxy

  • Install Copy the binaries binary as udmg-auth-proxy.exe

  • Create the configuration file C:\UDMG\UDMG Auth Proxy\config.toml with the following parameters:

Panel

# Proxy Configuration
[proxy]
# Port, default "5000"
port = "5000"
# Network interface, default "0.0.0.0"
inet = "127.0.0.1"
# Enable recover on panic, default true, should be true for production environment
recover = true
# Enable Cross-Origin Resource Sharing (CORS), should be true for production environment
cors = true
# CORS: List of origins that may access the resource. Optional. Default value "*"
# domain = "*"
# Enable Request Track ID, default true
tracker = true
# Enable Request Logger, default true
logger = true
# Rate Limit IP Request over 1 second, default 0 (unlimited)
limit = 0
# Enable the Prometheus Metric Endpoint '/metric', default false
metrics = false
# Enable CSRF token
csrf = false
# Service 'local' with direct authentication on the UDMG Server
the UDMG Server
[service.local]
# UDMG Server Listen Protocol
protocol = "http"
[[service.local.targets]]
# UDMG Server Listen Protocol
protocol = "http"
[[service.local.targets]]
# UDMG Server Hostname or IP
hostname = "localhost"
# UDMG Server Port
port = 18080

Please refer to Authentication Methods for the LDAP and SSO authentication options.

UDMG Agent Proxy

...

Hostname or IP
hostname = "localhost"
# UDMG Server Port
port = 18080

Please refer to Authentication Methods for the LDAP and SSO authentication options.


UDMG Agent Proxy


Agent Proxy Server Configuration

  • Create a directory C:\UDMG\UDMG Agent\agent

  • Copy the binary as udmg-agent-server.exe

Panel

C:\> ssh-keygen -t rsa -q -N "" -f "C:\UDMG\UDMG Agent

...

Install the binaries as udmg-agent-proxy-client.exe and udmg-agent-proxy-server.exe

Agent Proxy Server Configuration

Panel

# ssh-keygen -t rsa -q -N "" -f "\agent\agent_key"

If OpenSSH is not installed or not available, the PuTTY tool can be used instead.

Use PuTTYgen to generate a key pair for the agent, more detailed instructions can be found here: Using public keys for SSH authentication

Image Added

After generating the key, export it with OpenSSH format:

Image Added

  • Create a configuration file as C:\UDMG\UDMG Agent\agent

...

If OpenSSH is not installed or not available, the PuTTY tool can be used instead.

Use PuTTYgen to generate a key pair for the agent, more detailled instructions can be found here: Using public keys for SSH authentication

Image Removed

After generating the key, export it with OpenSSH format:

Image Removed

  • Create a configuration file as C:\UDMG\UDMG Agent\agent\agent.toml\agent.toml

Note
titleNote

The Windows paths in the TOML configuration file must be specified either in UNIX-style, using forward slashes '/', or with double backslashes '\\'.

For example: 

ssh_key = "C:/UDMG/UDMG Agent/agent/agent_key"

ssh_key = "C:\\UDMG\\UDMG Agent\\agent\\agent_key" 


Panel

[agent]
# Listen IP Address
hostname = "0.0.0.0"
# Listen Port
port = "2222"
# SSH Priv Key
ssh_key = "C:\/UDMG\/UDMG Agent\/agent/agent_key"
# SSH Public Key
ssh_key_pub = "C:\/UDMG\/UDMG Agent\/agent/agent_key.pub"

# Service User
username = "mft"
# Service Password
password = "61ee8b5601a84d5154387578466c8998848ba089"

...

Agent Proxy Client Configuration

  • Create a directory "C:\UDMG\UDMG Agent\client"

  • Copy the binary as udmg-agent-client.exe

  • Create a configuration file as "C:\UDMG\UDMG Agent\client\client.toml"

Panel

[client]
# UDMG Agent Proxy Hostname or IP
hostname = "localhost"
# UDMG Agent Proxy Listen Port
port = "2222"

# UDMG Agent Service User and password
username = "mft"
password = "61ee8b5601a84d5154387578466c8998848ba089"

# Default TTL to Connection Retry
ttl="5s"

# UDMG Agent Client Admin API
[client.api]
# Listen Port
port="2280"

# UDMG Server
[gateway]
# UDMG Server Hostname or IP
hostname = "localhost"
# UDMG Server Port
port = "18080"
# UDMG Server Username/Password
username = "admin"
password = "admin_password"

...

The UDMG components can be installed as Windows service with the WinSw WinSW tool.

NGINX Server

  • Download WinSW and copyy winsw.exe under C:\UDMG\nginx with the following name: nginx-service.exe
  • Create a new service definition file: nginx-service.yml

...

Panel

## UDMG server service configuration
##
## references:
## https://github.com/winsw/winsw/blob/v2.11.0/doc/yamlConfigFile.md
## https://github.com/winsw/winsw/blob/v2.11.0/examples/sample-allOption.yml

id: UDMG Server
name: UDMG Server
description: UDMG Server

## SECTION: Installation
## These options are being used during the installation only.
## Their modification will not take affect without the service re-installation.
##
## service account can be given during the installation with /p parameter
## waarp-gatewayd-service-user.exe install /p
## and then providing username
onFailure:
  -
    action: restart
    delay: 10 sec
  -
    action: restart
    delay: 60 sec
  -
    action: none
    #resetFailure: 01:00:00

## SECTION: Executable management
executable: C:\UDMG\UDMG Server\udmg-server.exe
startArguments: >
server
arguments: server -c "C:\UDMG\UDMG Server\server.ini"
workingdirectory: C:\UDMG\UDMG Server
priority: Normal

#stopTimeout: 15 sec
#stopParentProcessFirst: true

## SECTION: Service management
#startMode: Automatic
#delayedAutoStart: true

## SECTION:Logging
## Note: disabled due to WinSW bug https://github.com/winsw/winsw/issues/1016
#log:
# mode: roll-by-size-time
# sizeThreshold: 10240
# keepFiles: 8
# pattern: yyyyMMdd
# autoRollAtTime: 00:00:00

## SECTION: Environment setup

## SECTION: Extensions
## This configuration section allows specifying custom extensions.
## More info is available here: https://github.com/kohsuke/winsw/blob/v2.11.0/doc/extensions/extensions.md

...

Panel

C:\UDMG\UDMG Server>udmg-server-service.exe install
2022-10-07 10:07:48,385 INFO  - Installing service 'UDMG Server (UDMG Server)'...
2022-10-07 10:07:48,449 INFO  - Service 'UDMG Server (UDMG Server)' was installed successfully.

C:\UDMG\UDMG Server>udmg-server-service.exe status
Stopped

C:\UDMG\UDMG Server>udmg-server-service.exe start
2022-10-07 10:08:19,297 INFO  - Starting service 'UDMG Server (UDMG Server)'...
2022-10-07 10:08:20,418 INFO  - Service 'UDMG Server (UDMG Server)' started successfully.

C:\UDMG\UDMG Server>udmg-server-service.exe status
Started

Be sure that the UDMG Server API listen port (8080 by default) and network interface is are reachable by UDMG Authentication Proxy and UDMG Agent Client. Refer to Ports Configuration for more information.

UDMG Authentication Proxy

...

Panel

C:\UDMG\UDMG Auth Proxy>udmg-auth-proxy-service.exe install
2022-10-07 10:09:45,618 INFO  - Installing service 'UDMG Auth Proxy (UDMG Auth Proxy)'...
2022-10-07 10:09:45,683 INFO  - Service 'UDMG Auth Proxy (UDMG Auth Proxy)' was installed successfully.

C:\UDMG\UDMG Auth Proxy>udmg-auth-proxy-service.exe start
2022-10-07 10:09:52,852 INFO  - Starting service 'UDMG Auth Proxy (UDMG Auth Proxy)'...
2022-10-07 10:09:54,053 INFO  - Service 'UDMG Auth Proxy (UDMG Auth Proxy)' started successfully.

C:\UDMG\UDMG Auth Proxy>udmg-auth-proxy-service.exe status
Started

Be sure that the UDMG Authentication Proxy API listen port (5000 by default) and network interface is are reachable by NGINX ServerNGINX Server. Refer to Ports Configuration for more information.

UDMG Agent Proxy

Agent Proxy Server Service

  • Download WinSW and copy winsw.exe under C:\UDMG\UDMG Agent\agent with the following name: udmg-agent-proxy-server-service.exe
  • Create a new service definition file: udmg-agent-proxy-server-service.yml
Panel

id: UDMG Agent Server
name: UDMG Agent Server
description: UDMG Agent Server
executable: C:\UDMG\UDMG Agent\agent\udmg-agent-proxy-server.exe
priority: Normal
workingdirectory: C:\UDMG\UDMG Agent\agent
env:
- name: MFTUDMG_AGENT_PROXY_CONFIG
  value: 'C:\UDMG\UDMG Agent\agent\agent.toml'

...

Panel

C:\UDMG\UDMG Agent\agent>udmg-agent-proxy-server-service.exe install
2022-10-07 10:09:45,618 INFO  - Installing service 'UDMG Agent Server (UDMG Agent Server)'...
2022-10-07 10:09:45,683 INFO  - Service 'UDMG Agent Server (UDMG Agent Server)' was installed successfully.
 
C:\UDMG\UDMG Agent\agent>udmg-agent-proxy-server-service.exe start
2022-10-07 10:09:52,852 INFO  - Starting service 'UDMG Agent Server (UDMG Agent Server)'...
2022-10-07 10:09:54,053 INFO  - Service 'UDMG Agent Server (UDMG Agent Server)' started successfully.
 
C:\UDMG\UDMG Agent\agent>udmg-agent-proxy-server-service.exe status
Started

Be sure that the UDMG Agent Proxy Server listen port (2222 by default) and network interface is are reachable by UDMG Agent Client. Refer to Ports Configuration for more information.

Agent Proxy Client Service

  • Download WinSW and copy winsw.exe under C:\UDMG\UDMG Agent\client with the following name: udmg-agent-proxy-client-service.exe
  • Create a new service definition file: udmg-agent-proxy-client-service.yml
Panel

id: UDMG Agent Client
name: UDMG Agent Client
description: UDMG Agent Client
executable: C:\UDMG\UDMG Agent\client\udmg-agent-proxy-client.exe
priority: Normal
workingdirectory: C:\UDMG\UDMG Agent\client
env:
- name: MFTUDMG_AGENT_PROXY_CONFIG
  value: 'C:\UDMG\UDMG Agent\client\agentclient.toml'

  • Start the service and check the status:
Panel

C:\UDMG\UDMG Agent\client>udmg-agent-proxy-client-service.exe install
2022-10-07 10:09:45,618 INFO  - Installing service 'UDMG Agent Client(UDMG Agent Client)'...
2022-10-07 10:09:45,683 INFO  - Service 'UDMG Agent Client (UDMG Agent Client)' was installed successfully.

C:\UDMG\UDMG Agent\client>udmg-agent-proxy-client-service.exe start
2022-10-07 10:09:52,852 INFO  - Starting service 'UDMG Agent Client (UDMG Agent Client)'...
2022-10-07 10:09:54,053 INFO  - Service 'UDMG Agent Client(UDMG Agent Client)' started successfully.

C:\UDMG\UDMG Agent\client>udmg-agent-proxy-client-service.exe status
Started

...

Ports

...

Configuration

See Network Requirements.

References

This document references the following documents.

Name

Location

PostgreSQL Client Authentication

21.1. The pg_hba.conf File

PostgreSQL Password Authentication

21.5. Password Authentication

Guide on setting up Nginx as a service on Windowshttps://github.com/sheggi/win-service-nginx

...