...
The following illustration is a typical, although simplified, Universal Data Manager Gateway system in a High Availability environment.
...
The mode (status) of a cluster node indicates whether or not it is the cluster node that currently is processing work:
Active | Cluster node currently is performing all file transfers and the also the other system processing functions (administration, reporting, …). It supports the local services and the processing of transfers in server and client modes. |
---|---|
Passive | Cluster Node is not performing any file transfers but is available to perform the other system processing functions. |
Offline | Cluster node is not running or is inoperable and needs to be restarted. |
High Availability Start-Up
The following steps describe how a High Availability environment starts up:
Step 1 | User starts the Cluster Nodes. |
---|---|
Step 2 | Each cluster node reads its |
Step 3 | Each cluster node locates and connects to the database and retrieves information about the UDMG environment. |
Step 4 | One of the nodes becomes the Active node. It starts the local services (local FTP, SFTP, ... servers) and starts the processing of transfer requests, both in server mode and client mode. |
Note | ||
---|---|---|
| ||
Cluster nodes in Passive mode can perform limited system processing functions. |
Determining Mode of a Cluster Node at Start-up
A cluster node starts in Passive mode. It then determines if it should remain in Passive mode or switch to Active mode.
The following flow chart describes how a cluster node determines its mode at start-up:
Note | ||
---|---|---|
| ||
A cluster node is considered "healthy" or "stale" based on its heartbeat timestamp. |
Checking the Active Cluster Node During Operations
When all cluster nodes have started, each one continuously monitors the heartbeats of the other running cluster nodes.
If a Passive cluster node determines that the Active cluster node is no longer running, the Passive cluster node automatically takes over as the Active cluster node based upon the same criteria described above.
This determination is made as follows:
Step 1 | The Active cluster node sends a heartbeat by updating a timestamp in the database. |
---|
...
seconds |
...
. It can be adjusted with the 'Heartbeat' parameter:
| |
Step 2 | All Passive cluster nodes check the Active cluster node's timestamp to determine if it is current. |
---|
...
|
...
20 seconds. It can be adjusted with the 'HeartbeatCheck' parameter:
| |
Step 3 | If a Passive cluster node determines that the Active cluster node's timestamp is stale, failover occurs: the Passive cluster node changes the mode of the Active cluster node to Offline and takes over as the Active cluster node. If more than one cluster node is operating in Passive mode, the first cluster node eligible to become Active that determines that the Active cluster node is not running becomes the Active cluster node. A stale cluster node is one whose timestamp is older than 5 minutes. It can be adjusted with the 'Deadline' parameter:
|
---|
What To Do If a Failover Occurs
A Passive cluster node taking over as an Active cluster node is referred to as failover. If failover occurs, the event is invisible unless you are using the Active cluster node in a browser.
If you are using the Active cluster node in a browser and the cluster node fails, you will receive a browser error. In this case, take the following steps to continue working:
Step 1 | Access the new Active cluster node in your browser. This can be achieved by selecting the appropriate service in that UDMG Admin UI login page, provided that each UDMG Server instance is defined as a dedicated login service. |
---|---|
Step 2 | If you were adding, deleting, or updating records at the time of the failure, check the record you were working on. Any data you had not yet saved will be lost. |
Viewing Cluster Node Status
The cluster node status is displayed by the background color of the Server Status button.
...
It is also indicated by the "Controller" service Information string:
High Availability Configuration
To achieve High Availability for your Universal Data Mover Gateway, you must configure the UDMG Server cluster nodes, UDMG Authentication Proxy.
Configuring Cluster Nodes
All cluster nodes in a High Availability environment must point to the same database by making sure the following entries in their server.ini files are the same.
For example:
Code Block |
---|
[database] ; Type of the RDBMS used for the gateway database. Possible values: sqlite, mysql (default), postgresql, oracle, mssql Type = postgresql ; Address (URL:port) of the database. The default port depends on the type of database used (PostgreSQL: 5432, MySQL: 3306, MS SQL: 1433, Oracle: 1521, SQLite: none). Address = localhost:5432 ; The name of the database Name = udmg ; The name of the gateway database user User = udmg_user ; The password of the gateway database user Password = udmg_password ; Path of the database TLS certificate file (only supported for mysql, postgresql). ; TLSCert = ; Path of the key of the TLS certificate file (only supported for mysql, postgresql). ; TLSKey = ; The path to the file containing the passphrase used to encrypt account passwords using AES. Recommended to be a full absolute path, if the file does not exist, a new passphrase is generated the first time. AESPassphrase = /opt/udmg/etc/udmg-server/passphrase.aes |
The heartbeat frequency and failover deadline can also be tuned with the following parameters:
Code Block |
---|
[controller]
; The frequency at which the heartbeat will be updated
Heartbeat = 10s
; The deadline to determine if this instance will be active
Deadline = 5m
; The heartbeat to determine if this instance will be probed
HeartbeatCheck = 20s |
Load Balancer
REST API connections
If you are using a load balancer in your High Availability environment, it can utilize the following HTTP requests to direct the UDMG Admin UI and REST API requests to the active instance:
| If a cluster node is active, this URL returns the status |
...
200 (OK) and a simple one word content |
...
of ACTIVE. |
...
400 (cluster node is not active) and lists the actual mode of the cluster node: |
...
PASSIVE or |
...
OFFLINE. This API is provided without authentication. | |||
http(s)://serverhost:[Port]/api/sb_healthcheck | This URL returns information about a cluster node:
This API requires authentication but no specific permissions. | ||
http(s)://serverhost:[Port]/api/sb_mgmt_nodes | This URL returns information about all the cluster nodes:
This API requires authentication and the 'administration read' permission. |
This can be used to direct the UDMG Admin UI and REST API requests to the active instance.
Local MFT server connections
Until the node is marked as 'ACTIVE', the local server services that controlled by UDMG Server are not be started and there is no listener on their defined port.
As there can be some delay between the node status change and the actual start and stop of each local MFT server, it is recommended to use a TCP connection test to determine which node is currently owning the listener.
If you have the possibility of performing a double health check, you can choose to:
- Validate the node status endpoint
- Validate that the port is available
Example of a configuration with HAProxy:
Code Block |
---|
# --------------------------------------------------------------------------- #
# Configuration for: <service-name>
# --------------------------------------------------------------------------- #
frontend frontend-<service-name>
bind *:<service-port> transparent
mode tcp
log global
option tcplog
use_backend backend-<service-name>
backend backend-<service-name>
mode tcp
stick-table type ip size 10k expire 300s
stick on src
# Healtcheck
option httpchk
http-check send meth GET uri /ping
http-check expect status 200
# Pool Servers
server node-0 <hostname-node-0> check port <node-0-server-api-port> cookie S01 check inter 300s
server node-1 <hostname-node-1> check port <node-1-server-api-port> cookie S02 check inter 300s |