Disclaimer

Your use of this download is governed by Stonebranch Integration Hub - Terms of Use.

Overview

UDMG (Universal Data Mover Gateway) is a business-to-business MFT (Managed File Transfer) Solution. It provides the capabilities to securely manage File transfers with external business partners (i.e. Customers or Suppliers) and between any endpoint in today’s Hybrid-IT infrastructure, on-premises, or in the cloud (public and or private).

This Universal Extension provides the capability to perform file transfer where UDMG is the client and upload or download files to remote partners.

Version Information

Template Name	Extension Name	Version

Template Name	Extension Name	Version
UDMG File Transfer	ue-mft-transfer	1 (Current 1.4.1)

Refer to the changelog for version history information.

Key Features

Feature	Description

Feature	Description
Actions	GET: trigger a File Transfer on UDMG to download a file from a remote partner. PUT: trigger a File Transfer on UDMG to upload a file to a remote partner. MGET: trigger a Wildcard File Transfer on UDMG to download several files from a remote partner. MPUT: trigger a Wildcard File Transfer on UDMG to upload several files to a remote partner. LIST: trigger a Wildcard List File Transfer on UDMG to list the files matching a wildcard pattern on a remote SFTP partner.
File Transfer Control	Capability to pause a partial transfer. Capability to resume a paused transfer. Capability to cancel a transfer (from planned, running, paused, or error status). Capability to retry a failed transfer. Synchronous mode: the task monitors the UDMG transfer and terminates when the transfer is completed (either with success or error). Asynchronous mode: the task registers the UDMG transfer and terminates when the transfer request is acknowledged on UDMG.
Input/Output	Capability for the UAC task author, to retrieve the list of partners from UDMG for a given protocol. Capability for the UAC task author, to retrieve the list of transfer rules from UDMG for a given partner. Monitor the transfer with periodic polling from UDMG and display to output-only fields and extension output.
Other	Support for SFTP, HTTPS, PeSIT, and PeSIT-TLS file transfer protocols.

Software Requirements

This integration requires a Universal Agent and a Python runtime to execute the Universal Task.

Area	Details

Area	Details
Universal Agent & Python version	Python 3.7 and Agent Version >= 7.3.0.0. Python 3.11 and Agent Version >= 7.6.0.0.
Universal Controller	Universal Controller Version >= 7.3.0.0.
UDMG Server	UDMG Server 1.2.1.0 or higher.
Network and Connectivity	Universal Agent requires connectivity towards the UDMG Server REST API.

Configuration Examples


Scenario A Example for sending a file to a `HTTPS` server.	Scenario A The partner configuration on UDMG.

Scenario B Example of downloading a file from an `SFTP` server.	Scenario B The partner configuration on UDMG.

Scenario C Example of downloading multiple files from an `SFTP` server.	Scenario D Example of listing files from an `SFTP` server.

Action Output

Output Type	Description	Examples

Output Type

Description

Examples

EXTENSION

“exit_code“, “status“, “status_description“: General info regarding the task execution. In case of failure, it's set according to the specific error that happened during the execution. A list of possible values is provided in the exit codes table in this document.
“invocation” > “fields”: The task configuration used for this task execution.
“result“: For a comprehensive overview of the results, please refer to the table below.

Example of a Single download from SFTP

{
    "exit_code": 0,
    "status_description": "Task executed successfully",
    "invocation": {
        "fields": {
            "credentials_user": "admin",
            "credentials_password": "****",
            "action": "GET",
            "protocol": "sftp",
            "mft_server": "http://udmg.somehost.com:8080/api",
            "file": "readme.txt",
            "output": null,
            "partner": "rebex",
            "rule": "rebex_receive_example",
            "use_dynamic_partner_account": false,
            "partner_account_dynamic": null,
            "user": "demo",
            "wait_for_completion": true,
            "transfer_date": null,
            "polling_interval": 5,
            "max_file_to_monitor": 100,
            "ue_task_uuid": "16835525405027449004GAT04TVODUWH"
        }
    },
    "result": {
        "mft_transfer": {
            "transfer_id": "478",
            "start": "2023-05-11 10:04:36.433385+00:00",
            "stop": "2023-05-11 10:04:40.908049+00:00",
            "status": "DONE",
            "step": "StepNone",
            "transfer_progress": "405  Bytes",
            "task_number": null,
            "error_code": "TeOk",
            "error_msg": null,
            "local_file": "/data/in/readme.txt",
            "remote_file": "/pub/example/readme.txt",
            "transfer_uuid": "1656601189657083904",
            "transfer_info": {
                "udmg_file_computed_extension": ".txt",
                "udmg_file_computed_mimetype": "text/plain; charset=utf-8",
                "udmg_sftp_client_version": "SSH-2.0-Go",
                "udmg_sftp_config_ciphers": [
                    "aes128-gcm@openssh.com",
                    "chacha20-poly1305@openssh.com",
                    "aes128-ctr",
                    "aes192-ctr",
                    "aes256-ctr"
                ],
                "udmg_sftp_config_kex": [
                    "curve25519-sha256@libssh.org",
                    "ecdh-sha2-nistp256",
                    "ecdh-sha2-nistp384",
                    "ecdh-sha2-nistp521",
                    "diffie-hellman-group-exchange-sha256",
                    "diffie-hellman-group1-sha1",
                    "diffie-hellman-group14-sha1"
                ],
                "udmg_sftp_config_mac": [
                    "hmac-sha2-256-etm@openssh.com",
                    "hmac-sha2-256",
                    "hmac-sha1",
                    "hmac-sha1-96"
                ],
                "udmg_sftp_failed_auth_methods": [
                    "none"
                ],
                "udmg_sftp_hostkey_algo": "ssh-rsa",
                "udmg_sftp_kex_algo": "curve25519-sha256@libssh.org",
                "udmg_sftp_read_algos": {
                    "cipher": "aes128-gcm@openssh.com",
                    "compression": "none",
                    "mac": ""
                },
                "udmg_sftp_server_version": "SSH-2.0-RebexSSH_5.0.8466.0",
                "udmg_sftp_sucessful_auth_methods": "password",
                "udmg_sftp_write_algos": {
                    "cipher": "aes128-gcm@openssh.com",
                    "compression": "none",
                    "mac": ""
                },
                "udmg_ue_task_uuid": "16835525405027449004GAT04TVODUWH",
                "udmg_xfer_log": "/config/logs/478.log"
            }
        },
        "mft_transfer_files_summary": {
            "done": 1,
            "error": 0,
            "cancelled": 0
        },
        "errors": []
    }
}

Example of a Wildcard download from SFTP with 3 matching files

{
    "exit_code": 0,
    "status_description": "Task executed successfully",
    "invocation": {
        "fields": {
            "credentials_user": "admin",
            "credentials_password": "****",
            "action": "MGET",
            "protocol": "sftp",
            "mft_server": "http://udmg.somehost.com:8080/api",
            "file": "m*.png",
            "output": null,
            "partner": "rebex",
            "rule": "rebex_receive_example",
            "use_dynamic_partner_account": false,
            "partner_account_dynamic": null,
            "user": "demo",
            "wait_for_completion": true,
            "transfer_date": null,
            "polling_interval": 5,
            "max_file_to_monitor": 100,
            "ue_task_uuid": "168355254050277290080KCA7YT971DJ"
        }
    },
    "result": {
        "mft_transfer": {
            "transfer_id": "479",
            "start": "2023-05-11 10:44:30.493303+00:00",
            "stop": "2023-05-11 10:44:31.144255+00:00",
            "status": "DONE",
            "step": "StepNone",
            "transfer_progress": "3 out of 3 files",
            "task_number": null,
            "error_code": "TeOk",
            "error_msg": null,
            "local_file": "/data/in/m*.png",
            "remote_file": "/pub/example/m*.png",
            "transfer_uuid": "1656611231072518144",
            "transfer_info": {
                "udmg_sftp_client_version": "SSH-2.0-Go",
                "udmg_sftp_config_ciphers": [
                    "aes128-gcm@openssh.com",
                    "chacha20-poly1305@openssh.com",
                    "aes128-ctr",
                    "aes192-ctr",
                    "aes256-ctr"
                ],
                "udmg_sftp_config_kex": [
                    "curve25519-sha256@libssh.org",
                    "ecdh-sha2-nistp256",
                    "ecdh-sha2-nistp384",
                    "ecdh-sha2-nistp521",
                    "diffie-hellman-group-exchange-sha256",
                    "diffie-hellman-group1-sha1",
                    "diffie-hellman-group14-sha1"
                ],
                "udmg_sftp_config_mac": [
                    "hmac-sha2-256-etm@openssh.com",
                    "hmac-sha2-256",
                    "hmac-sha1",
                    "hmac-sha1-96"
                ],
                "udmg_sftp_failed_auth_methods": [
                    "none"
                ],
                "udmg_sftp_hostkey_algo": "ssh-rsa",
                "udmg_sftp_kex_algo": "curve25519-sha256@libssh.org",
                "udmg_sftp_read_algos": {
                    "cipher": "aes128-gcm@openssh.com",
                    "compression": "none",
                    "mac": ""
                },
                "udmg_sftp_server_version": "SSH-2.0-RebexSSH_5.0.8466.0",
                "udmg_sftp_sucessful_auth_methods": "password",
                "udmg_sftp_write_algos": {
                    "cipher": "aes128-gcm@openssh.com",
                    "compression": "none",
                    "mac": ""
                },
                "udmg_ue_task_uuid": "168355254050277290080KCA7YT971DJ",
                "udmg_xfer_childIDs": [
                    480,
                    481,
                    482
                ],
                "udmg_xfer_files": [
                    "mail-editor.png",
                    "mail-send-winforms.png",
                    "mime-explorer.png"
                ],
                "udmg_xfer_log": "/config/logs/479.log",
                "udmg_xfer_wildcard": true
            }
        },
        "mft_transfer_files_summary": {
            "done": 3,
            "error": 0,
            "cancelled": 0
        },
        "errors": []
    }
}

Example of a wildcard list from SFTP with 2 matching files

{
    "exit_code": 0,
    "status": "SUCCESS",    
    "status_description": "SUCCESS",
    "invocation": {
        "fields": { ... }
    },
    "result": {
        "mft_transfer": {
            "transfer_id": "145",
            "start": "2023-05-26T12:36:36.405685+00:00",
            "stop": "2023-05-26T12:36:37.022167",
            "status": "DONE",
            "step": "StepNone",
            "progress": 2,
            "task_number": null,
            "error_code": "TeOk",
            "error_msg": null,
            "local_file": "/home/udmg/udmg-server/in/wince*.png",
            "remote_file": "/pub/example/wince*.png",
            "transfer_uuid": "1662075259576123392",
            "transfer_info": {
                "udmg_sftp_hostkey_algo": "ssh-rsa",
                "udmg_sftp_kex_algo": "curve25519-sha256@libssh.org",
                "udmg_sftp_read_algos": {
                    "cipher": "aes128-gcm@openssh.com",
                    "compression": "none",
                    "mac": ""
                },
                "udmg_sftp_server_version": "SSH-2.0-RebexSSH_5.0.8466.0",
                "udmg_sftp_sucessful_auth_methods": "password",
                "udmg_sftp_write_algos": {
                    "cipher": "aes128-gcm@openssh.com",
                    "compression": "none",
                    "mac": ""
                },
                "udmg_xfer_files": [
					"winceclient.png",
					"winceclientSmall.png"
                ],                
                "udmg_xfer_log": "/home/mft-user/waarp/log/145.log",
                "udmg_xfer_wildcard": true,
                "udmg_xfer_cmd": "list",
                "udmg_ue_task_uuid": "8a801e6d164c4648a32f94dbcd1bb19a"
            }
        }
    }
}

The Extension output contains the attribute result which contains the following sub-attributes:

Attribute	Type	Description	Version Information

Attribute	Type	Description	Version Information
mft_transfer	object	The details of the main transfer record (the single transfer or the parent transfer in case of Wildcard Transfer).	Introduced in 1.0.0
mft_transfer.transfer_id	string	The identifier of the transfer on UDMG.	Introduced in 1.0.0
mft_transfer.status	string	The current status of the transfer One of PLANNED, RUNNING, PAUSED, INTERRUPTED, DONE, CANCELED or ERROR.	Introduced in 1.0.0
mft_transfer.start	string	The date and time the transfer has started (UDMG Server time)	Introduced in 1.0.0
mft_transfer.stop	string	The date and time the transfer has ended (UDMG Server time). For a wildcard transfer, this is when the listing of matching files and the registration of the child transfers is completed.	Introduced in 1.0.0
mft_transfer.step	string	The current processing step of the transfer One of NONE, PRE TASKS, DATA, POST TASKS, ERROR TASKS or FINALIZATION.	Introduced in 1.0.0
mft_transfer.transfer_progress	text	The progress of the data transfer in human-readable format size in KB, MB, or GB for a Single Transfer, "x out of y files" for a Wildcard Transfer.	Introduced in 1.0.0
mft_transfer.error_code	string	The error code of the transfer (if an error occurred).	Introduced in 1.0.0
mft_transfer.error_msg	string	The error message (if an error occurred).	Introduced in 1.0.0
mft_transfer.task_number	string	The task number in the processing step. Only for the steps 'PRE TASKS', 'POST TASKS', and 'ERROR TASKS'.	Introduced in 1.0.0
mft_transfer.local_file	string	Local file path.	Introduced in 1.0.0
mft_transfer.remote_file	string	Remote file path.	Introduced in 1.0.0
mft_transfer.transfer_uuid	string	The UUID of the transfer on UDMG.	Introduced in 1.1.0
mft_transfer.transfer_info	object	The transfer information fields. Set of metadata associated with the transfer on the UDMG Server. Fields with the udmg_ prefix are set internally by the UDMG server. Fields with the udmg_ue prefix are set by the UDMG File Transfer task.	Introduced in 1.1.0
mft_transfer.transfer_info.udmg_ue_task_uuid	string	The UUID of the UDMG File Transfer task. It can be used to easily correlate the UC task executions and the corresponding UDMG file transfers.	Introduced in 1.2.0
mft_transfer.transfer_info.udmg_xfer_wildcard	boolean	Indicator that this is a wildcard transfer. Only for MPUT and MGET.	Introduced in 1.2.0
mft_transfer.transfer_info.udmg_xfer_cmd	string	'`list`' is an indicator that this is a wildcard list command. Only for LIST.	Introduced in 1.3.0
mft_transfer.transfer_info.udmg_xfer_childIDs	array	A list of UDMG transfer IDs that are created in the context of this file file transfer. For each file matching the filename input, one file transfer process is created in UDMG. Only for MPUT and MGET.	Introduced in 1.2.0
mft_transfer.transfer_info.udmg_xfer_files	array	The list of files that matched the user input. Each of these files is transferred with a separate file transfer execution, which transfer IDs can be found in “udmg_xfer_childIDs“ list. Only for MPUT and MGET.	Introduced in 1.2.0
mft_transfer_files_summary	object	Summary counters	Introduced in 1.2.0
mft_transfer_files_summary.done	number	Number of file transfers with success.	Introduced in 1.0.0
mft_transfer_files_summary.error	number	The number of file transfers in error state.	Introduced in 1.0.0
mft_transfer_files_summary.cancelled	number	The number of file transfers that are canceled.	Introduced in 1.0.0
errors	array	List of errors for the child transfers.	Introduced in 1.2.0

Input Fields

Field	Type	Mandatory	Description	Version Information

Field	Type	Mandatory	Description	Version Information
Action	Choice	Yes	Action performed upon the task execution. Available actions: GET PUT (Default) MGET MPUT LIST	Introduced in 1.0.0
Protocol	Choice	Yes	The transfer protocol. Available values: SFTP (Default) HTTP HTTPS PeSIT PeSIT-TLS Note The protocol serves as a filter for the Partner list. The actual protocol for the file transfer is the one that is configured on UDMG for the selected Partner	Introduced in 1.0.0
MFT Server	Text	Yes	UDMG Server URL. This is the URL for the REST API endpoint, the port is configured in the UDMG Server configuration file. For example: http://udmg.company.com:8080/api	Introduced in 1.0.0
MFT Credentials	Credentials	Yes	UDMG Server Credentials. The Credentials definition should be as follows. User Name as "Runtime User". User Password as "Runtime Password". The user must have read permissions for rules and partners, read/write permission for transfers	Introduced in 1.0.0
Source File	Text	Yes	The name of the source file, or the search pattern: PUT: local file GET: remote file MPUT: wildcard pattern for local files MGET: wildcard pattern for remote files LIST: wildcard pattern for remote files	Introduced in 1.0.0
Target File	Text	No	The name of the file on the target system after the transfer, PUT: remote file GET: local file MPUT/MGET/LIST: not used, the field is not displayed.	Introduced in 1.1.0
Partner	Dynamic Choice	Yes	Dynamically fetched list of partner names. The list is restricted to the partners that support the selected Protocol. The user can select the required partner name from a drop-down list.	Introduced in 1.0.0
Rule	Dynamic Choice	Yes	Dynamically fetched list of transfer rules. The list is restricted to the rules that are assigned to the selected Partner. The user can select the required rule from a drop-down list.	Introduced in 1.0.0
Dynamic Retrieval of Partner Account Options	Boolean	No	If enabled, the partner account field will be replaced by a dynamic choice field variant. The default value is false.	Introduced in 1.4.0
Partner Account	Text	Yes	The name of the account to use for the connection to the selected Partner. The account credentials are configured on UDMG. The field is visible when Dynamic Retrieval of Partner Account Options is false.	Introduced in 1.0.0
Partner Account	Dynamic Choice	Yes	The name of the account to use for the connection to the selected Partner. The account credentials are configured on UDMG. The field is visible when Dynamic Retrieval of Partner Account Options is true.	Introduced in 1.4.0
Wait for Transfer Completion	Boolean	Yes	By checking this field, the extension monitors the completion of the transfer on UDMG before completing its execution. Actions PUT/GET: Wait for the single transfer to be completed. Actions MPUT/MGET/LIST: Wait for the child file transfers to be completed or not. The extension will always wait for the parent process to be completed successfully, regardless of this configuration attribute. The default value is true.	Introduced in 1.2.0
Polling Interval	Integer	No	The amount of time (in seconds) the task will wait before polling to check the transfer status. Valid values start from 1. The default value is 5.	Introduced in 1.0.0
Max Files to Monitor

UDMG File Transfer