WARNING: Transaction error, attempt: %v of %v"    
ERROR:   Transaction error - MaxRetries reach %v of %v

Improve the validation of login provider JSON files during startup, invalid configuration is reported and the processs stops.

The error was previously reported as "[PANIC RECOVER] runtime error" during login attempts and not easily catchable as a configuration error.

Example of startup error:

level=error TS=2024-04-30T11:28:48.202909409Z Configuration=Auth Service=udmg_openldaps Error="ldap: read ca file: open ca-cert.pem: no such file or directory"
level=error TS=2024-04-30T11:28:48.202945585Z Error="Please validate you configuration file"

High Availability: Active/Passive instances with dynamic handover

Support for cluster awareness where multiple udmg-server instances are coordinating to have 1 active instance and the remaining ones remain in passive state. Continuous monitoring is performed to ensure that a passive instance will takeover in case of failure or shutdown of the active instance.

The MFT servers are owned by the active instance, which also manage the transfer requests in client mode.

See High Availability Behavior and Processing

Web services for the multi instance monitoring and load balancing.

• Existing /api/sb_healthcheck is enhanced with additional information: node details and node status. 
  Requires authentication but no specific permissions.
  Example of output for a passive instance:
  
  {

      "status": "operational",

      "nodeId": "gateway_1:8080-mft-gw-0",

      "nodeHostname": "gateway_1",

      "nodeIPAddress": "172.99.0.101",

      "nodePort": "8080",

      "nodeStatus": "PASSIVE",

      "nodeUptime": "15h6m4.810854748s",

      "nodeLastUpdate": "2023-11-14T08:07:54.848374Z",

      "nodeLastActiveDate": "2023-11-13T16:57:57.026091Z"

  }
• New /ping endpoint returns the plain text status: 'ACTIVE', 'PASSIVE', or 'OFFLINE' and also the 403 (Forbidden) HTTP code when the node is not active. This API is provided without authentication for load balancers.

• New /api/sb_mgmt_nodes endpoint returns the list of the instances with their details and status.
  Requires the 'administration read' permission.
  Example of output for 2 instances:
  

  { "nodes": [ { "nodeId": "gateway:8080-mft-gw-0", "nodeHostname": "gateway", "nodeIPAddress": "172.99.0.100", "nodePort": "8080", "nodeStatus": "ACTIVE", "nodeUptime": "16m15.413255244s", "nodeLastUpdate": "2023-11-09T15:24:20.562225Z", "nodeLastActiveDate": "2023-11-09T15:08:40.105002Z" }, { "nodeId": "gateway_1:8080-mft-gw-0", "nodeHostname": "gateway_1", "nodeIPAddress": "172.99.0.101", "nodePort": "8080", "nodeStatus": "PASSIVE", "nodeUptime": "16m0.631810433s", "nodeLastUpdate": "2023-11-09T15:24:35.289412Z", "nodeLastActiveDate": "2023-11-09T14:40:28.491424Z" } ] }
See REST API connections

CLI: Management of the 'administration' user permission. 

• Administration permissions are displayed on 'user list' or 'user get' commands
• read/write/delete can be assigned with the 'A' tag for administration permission on 'user add' and 'user update' commands
• 'superuser' shortcut is provided to create or convert a user to a superuser with all possible permissions, including the administration permissions
  

  udmg-client user update bob -r 'superuser' 

Improved usability of the Linux services:

• Service description is prefixed with "Stonebranch" for parity with Universal Agent ubroker service.
• Syslog messages are labeled with the service name (udmg-server, udmg-auth-proxy, ...) instead of 'sh'.
• Syslog messages are sent to 'local0' utility for easier filtering.

Allow absolute paths in SFTP client requests. If the remote filename stub in the transfer request starts with a '/' it is considered an absolute path and will be used without prefixing with the transfer rule remote directory path. 

For example, the client transfer udmg-client transfer add -f localFile.txt -o /remotepath/remoteFile.txt -p SftpPartner -l remoteAccount -w send -r Rule1S will now send /remotepath/remoteFile.txt as the path in the SFTP request whereas before, the path would have been {rule remote_directory}/other/remoteFile.txt.

Business Service as Tag

Improved the usability of Business Service : they can be used for tagging objects and not only to enforce security permissions. A Business Service without any User Group can be attached to an object as a way labeling it.

Permissions over the new business service are not required anymore when assigning an object to a business service thus allowing them to be used as tags (i.e. to group resources). The permissions required to assign an object to a business service are either: having the corresponding global permission (object write at user level) or having the corresponding business service permission (object write at user group level) in at least one of the object's business services.

REST API changes:

• Added new endpoints to add business server members:

• Added new endpoints to delete business server members:

• Added new endpoints to add user group members:

• Added new endpoints to delete user group members:

• Added new endpoints to list business server members:

Easier upgrade procedure, the migrate command now uses the last version by default.

$ /opt/udmg/bin/udmg-server migrate -l -c /opt/udmg/etc/udmg-server/server.ini | tail -1
1.5.0
$ /opt/udmg/bin/udmg-server migrate -c /opt/udmg/etc/udmg-server/server.ini -v -v -v
2023/10/30 10:57:24 [INFO    ] Migration: Starting upgrade migration...
...
2023/10/30 10:57:25 [INFO    ] Migration: Applying migration 'Bump database version to 1.5.0'

Disable Rule Task type

Ability to globally disable each rule task type.

New udmg-server configuration parameters, in the [tasks] section, for each task type allow to disable the related task processing during any file transfer. For the following task types: COPY, COPYRENAME, MOVE, MOVERENAME, DELETE, RENAME, EXEC, EXECMOVE, EXECOUTPUT, TRANSFER, CHECKREGEX, PUBLISHEVENT, ICAP. 

REST and CLI support for Personal Access Token.

Note that a token owner is always allowed to manage its own tokens.

New Web Services:

• /api/sb_users/{user}/tokens
  • GET to fetch the list of token: User read.
  • POST to create a token: User write.

• /api/sb_users/{user}/tokens/{token}
  • GET to read a given token: User read.
  • DELETE to remove a given token: User delete.

New CLI commands:

Usage:
  udmg-client [CONNECTION-OPTIONS] token personal [username] <command>

[personal command arguments]
  username:           The user's name

Available commands:
  add        Add a new personal access token to a user
  delete     Delete a personal access token
  get        Retrieve a personal access token
  list       List personal access tokens for a user

REST/CLI: User Profile

New web service and CLI command for a user to fetch its own user profile with system permissions.

GET /api/user_profile

• the output is the same as GET /api/sb_users/{user} for the calling user
• no permission is required

udmg-client user profile

Send transfer without pre-existing file

Allow a client send transfer to start even when the local file does not exist. It is now possible to start a send transfer even if the file to send does not yet exist, as long as it is created before the start of the data sending phase. Typically, this allows start a transfer where the file is created via rule pre-tasks.

Fix the transfer resume for SFTP Partner (client transfer), in some cases the resume operation would not continue the transfer from the last point.

The behavior is now:

• if the remote server file size is same than the transfer progress : resume from that point
• if the remote server file size is different from the transfer progress : stop transfer with ERROR
• If the remote target file is not found : resume the transfer from 0 (send whole file again)

Fix the missing allowed values for 'transfer list' command:  'ERROR', 'DONE', 'CANCELLED' statuses were missing.

$ udmg-client transfer list -h | grep status
      -t, --status=[PLANNED|RUNNING|INTERRUPTED|PAUSED|ERROR|DONE|CANCELLED] Filter

udmg-client user password bob -o ws -p ws

Append the instance hostname to global log file (when configured as a file in the configuration parameter LogTo. This enables multiple instances to share same log file storage.