Versions Compared

Old Version 1

changes.mady.by.user Stonebranch

Saved on

compared with

New Version Current

changes.mady.by.user Stonebranch

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Panel

Table of Contents

Requirements

System Requirements

  • Linux x64 (kernel 3.10 and later, glibc 2.17-78 and later), x86_64 based or Debian based

  • NGINX web server (1.20 and later)

  • PostgreSQL database (13 and later)

  • UDMG distribution files for the different modules:

    • UDMG Admin UI

    • UDMG Authentication Proxy

    • UDMG Server (Waarp Gateway)

    • UDMG Agent Proxy, this module is optional 

  • An account with administrative privileges for the installation.
  • Network connectivity via TCP/IP.
  • Approximately 100 megabytes of disk space for the installation. More disk space is required for log files.
  • A dedicated account for the execution of the UDMG components, with write access to the storage location for the files that will be managed by UDMG.

Database Space Requirements

Following the initialization of the Universal Data Mover Gateway database, the initial table space size will be approximately 10MB.

Based on calculations using data from all transfer types, each file transfer consumes approximately 2KB of database space. You should estimate space requirements for your data based on your expected number of file transfers per day and the duration for retaining history and activity data before purging.

Installing and Configuring the Components

PostgreSQL Database

  1. Create a blank database on the server. An already existing database can be used, but this is not recommended.

  2. Add a user that will be used as the owner of the related tables but also to authenticate with the server.
    In order to install or perform upgrades, this database user will require DDL (Data Definition Language) permission in the database during the installation or the upgrade.
    Once the install or upgrade has been completed successfully, the configured database user requires only DML (Data Manipulation Language) permissions.

Here is how to configure the database for a local installation where the database server is on the same host. For multi-node installation please refer to your database administrator.

Note

The following steps require root privilege, be sure that you have the correct access before to continue.

  • Install a PostgreSQL database server.

  • Once the installation is complete, initialize the PostgreSQL database.

  • Start the PostgreSQL Server:

Panel

# systemctl start postgresql

  • Create a user for UDMG Waarp Gateway

Login as PostgreSQL administrative user, start the PostgreSQL Console (psql) and create the database user for UDMG:

Panel

# sudo su - postgres
$ psql
psql (14.3)
Type "help" for help.

postgres=#

create database mft_waarp_gateway;
create user mft_waarp_gateway_user with encrypted password 'mft_waarp_gateway_password';
grant all privileges on database mft_waarp_gateway to mft_waarp_gateway_user;

  • Finally change the pg_hba.conf, to allow database connection with password. For example, for a system where the database server is on the same host as the UDMG server, by changing this line from:
Panel

host all all 127.0.0.1/32 ident

to:

Panel

host all all 127.0.0.1/32 md5

The exact configuration depends on the OS and database version, on the preferred security settings, and on the system architecture.

Panel

The location of the pg_hba.conf can be returned by psql:
$ sudo su - postgres
$ psql
postgres=# SHOW hba_file;

NGINX Server

Note

The following steps require root privilege, be sure that you have the correct access before to continue.

  • Run the following command to check the main NGINX configuration file:

Panel

# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

If the NGINX configuration file supports include directives, there will be a line like this:

Panel

include somedir/*.conf

  • Create a configuration file mft.conf under this directory (normally /etc/nginx/conf.d): This will create 2 location ‘/' for the UDMG Admin UI and '/service' for the internal authentication mechanism.

...

server {
    listen        8080;
    server_name   localhost;

    access_log    /var/log/nginx/host.access.log;

    location /service/ {
      proxy_pass  http://mft_auth_proxy/;
    }

    location / {
        try_files $uri $uri/ /index.html;
        root      /srv/www/mft;
    }
}

  • Validate that the configuration is correct with the following command:
Panel

# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

  • Create the Root directory:
Panel

# mkdir -p /srv/www/mft

  • Start the NGINX service using the Init system.
Panel

# systemctl start nginx

  • Check that the HTTP server was started and is running, for example with the curl command:
Panel

# curl http://localhost:8080
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.21.6</center>
</body>
</html>

This error (403) is excepted, since we don't have any asset deployed.

...

Installing and Configuring the Components

UDMG Admin UI

Note

The following steps require root privilege, be sure that you have the correct access before to continue.

  • Extract the distribution file for UDMG Admin UI, under the directory web server root directory, see the NGINX Service configuration above.

Panel

# tar xvf unzip -d /srv/www/mft/ udmg_admin_ui-<VERSION>.zip -C /srv/www/mft/

  • Validate that the service is working properly:

...

Panel

[global]
; The name given to identify this gateway instance. If the the database is shared between multiple gateways, this name MUST be unique across these gateways.
GatewayName = mft_waarp_gateway

; Default OS permission for created files
; FilePermissions = 700

; Default OS permission for created directories
; DirPermissions = 750


[paths]
; The root directory of the gateway. By default, it is the working directory of the process.
GatewayHome = /home/mft

; The directory for all incoming files.
; DefaultInDir = in

; The directory for all outgoing files.
; DefaultOutDir = out

; The directory for all running transfer files.
; DefaultTmpDir = tmp

[log]
; All messages with a severity above this level will be logged. Possible values are DEBUG, INFO, WARNING, ERROR and CRITICAL.
Level = DEBUG

; The path to the file where the logs must be written. Special values 'stdout' and 'syslog' log respectively to the standard output and to the syslog daemon
; LogTo = stdout

; If LogTo is set on 'syslog', the logs will be written to this facility.
; SyslogFacility = local0

[admin]
; The address used by the admin interface.
Host = 0.0.0.0

; The port used by the admin interface. If the port is 0, a free port will automatically be chosen.
Port = 18080

; Path of the TLS certificate for the admin interface.
; TLSCert =

; Path of the key of the TLS certificate.
; TLSKey =

[database]
; Name of the RDBMS used for the gateway database. Possible values: sqlite, mysql, postgresql
Type = postgresql

; Address of the database
Address = localhost

; The name of the database
Name = mft_waarp_gateway

; The name of the gateway database user
User = mft_waarp_gateway_user

; The password of the gateway database user
Password = mft_waarp_gateway_password

; Path of the database TLS certificate file.
; TLSCert =

; Path of the key of the TLS certificate file.
; TLSKey =

; The path to the file containing the passphrase used to encrypt account passwords using AES
; AESPassphrase = passphrase.aes

[controller]
; The frequency at which the database will be probed for new transfers
Delay = 300s

; The maximum number of concurrent incoming transfers allowed on the gateway (0 = unlimited).
; MaxTransferIn = 0

; The maximum number of concurrent outgoing transfers allowed on the gateway (0 = unlimited).
; MaxTransferOut = 0

[sftp]
; Set to true to allow legacy and weak cipher algorithms: 3des-cbc, aes128-cbc, arcfour, arcfour128, arcfour256
; AllowLegacyCiphers = false

...

Panel

# Proxy Configuration
[proxy]
# Port, default "5000"
port = "5000"
# Network interface, default "0.0.0.0"
inet = "127.0.0.1"
# Enable recover on panic, default true, should be true for production environment
recover = true
# Enable Cross-Origin Resource Sharing (CORS), should be true for production environment
cors = true
# Enable Request Track ID, default true
tracker = true
# Enable Request Logguer, default true
logger = true
# Rate Limit IP Request over 1 second, default 0 (unlimited)
limit = 0
# Enable the Prometheus Metric Endpoint '/metric', default false
metrics = false

# Service 'local' with direct authentication on the waarp gateway
[service.local]
# MFT Waarp Gateway Listen Protocol
protocol = "http"

[[service.local.targets]]
# MFT Waarp Gateway Hostname or IP
hostname = "localhost"
# MFT Waarp Gateway Port
port = 18080

# Service 'mft' with direct authentication on the waarp gateway
[service.mft]
# MFT Waarp Gateway Listen Protocol
protocol = "http"

[[service.mft.targets]]
# MFT Waarp Gateway Hostname or IP
hostname = "localhost"
# MFT Waarp Gateway Port
port = 18080

  • Install the binary under /usr/local/bin:

...

Panel

[agent]
# MFT Agent Proxy Hostname or IP, and port
hostname = "0.0.0.0"
port = "2222"
# path to the SSH private key file
ssh_key = "/etc/mft/agent_proxy/agent"
# path to the SSH public key file
ssh_key_pub = "/etc/mft/agent_proxy/agent.pub"

# Agent Service User and password
username = "mft"
password = "61ee8b5601a84d5154387578466c8998848ba089"

...

  • Enable the new service:
Panel

# systemctl enable mft_waarp_gateway.service
Created symlink /etc/systemd/system/multi-user.target.wants/mft_waarp_gateway.service → /etc/systemd/system/mft_waarp_gateway.service.

  • Start the service and check the status:

...