UDMG File Transfer
Disclaimer
Your use of this download is governed by Stonebranch’s Terms of Use, which are available at https://www.stonebranch.com/integration-hub/Terms-and-Privacy/Terms-of-Use/
Overview
UDMG (Universal Data Mover Gateway) is a business-to-business MFT (Managed File Transfer) Solution. It provides the capabilities to securely manage File transfers with external business partners (i.e. Customers or Suppliers), and between any endpoint in today’s Hybrid-IT infrastructure, on-premises, or in the cloud (public and or private).
This Universal Extension provides the capability to perform file transfer where UDMG is the client and upload or download files to remote partners.
Version Information
Template Name | Extension Name | Extension Version |
---|---|---|
UDMG File Transfer | ue-mft-transfer | 1.2.0 |
Refer to UDMG File Transfer#Changelog for version history information.
Software Requirements
This integration requires a Universal Agent and a Python runtime to execute the Universal Task.
Software Requirements for Universal Template and Universal Task
Requires Python 3.7.0 or higher. Tested with the Universal Agent bundled Python distribution.
Software Requirements for Universal Agent
Both Windows and Linux agents are supported.
- Universal Agent for Windows x64 Version 7.3.0.0 and later with python options installed.
- Universal Agent for Linux Version 7.3.0.0 and later with python options installed.
Software Requirements for Universal Controller
Universal Controller Version 7.3.0.0 and later.
Network and Connectivity Requirements
Universal Agent requires connectivity towards the UDMG Server REST API.
Key Features
This Universal Extension provides the following key features:
- Actions
- GET: trigger a File Transfer on UDMG to download a file from a remote partner.
- PUT: trigger a File Transfer on UDMG to upload a file to a remote partner.
- MGET:trigger a Wildcard File Transfer on UDMG to download several files from a remote partner.
- MPUT: trigger a Wildcard File Transfer on UDMG to upload several files to a remote partner.
- File Transfer Control
- Capability to pause a partial transfer.
- Capability to resume a paused transfer.
- Capability to cancel a transfer (from planned, running, paused, or error status).
- Capability to retry a failed transfer.
- Synchronous mode: the task monitors the UDMG transfer and terminates when the transfer is completed (either with success or error).
- Asynchronous mode: the task registers the UDMG transfer and terminates when the transfer request is acknowledged on UDMG.
- Input/Output
- Capability for the UAC task author, to retrieve the list of partners from UDMG for a given protocol.
- Capability for the UAC task author, to retrieve the list of transfer rules from UDMG for a given partner.
- Monitor of the transfer with periodic polling from UDMG and display to output-only fields and extension output.
- Other
- Support for SFTP, HTTPS, PeSIT, and PeSIT-TLS file transfer protocol.
Import Universal Template
To use the Universal Template, you first must perform the following steps.
This Universal Task requires the Resolvable Credentials feature. Check that the Resolvable Credentials Permitted system property has been set to true.
To import the Universal Template into your Controller, follow the instructions here.
When the files have been imported successfully, refresh the Universal Templates list; the Universal Template will appear on the list.
Modifications of this integration, applied by users or customers, before or after import, might affect the supportability of this integration. For more information refer to UDMG File Transfer#IntegrationModifications.
Configure Universal Task
For a new Universal Task, create a new task, and enter the required input fields.
Input Fields
The input fields for this Universal Extension are described below.
Field | Input type | Default value | Type | Description |
---|---|---|---|---|
Action | Required | PUT | Choice | Action performed upon the task execution. Available actions:
|
Protocol | Required | SFTP | Choice | The transfer protocol. Available values:
Note The protocol serves as a filter for the Partner list. The actual protocol for the file transfer is the one that is configured on UDMG for the selected Partner |
MFT Server | Required | - | Text | UDMG Server URL. This is the URL for the REST API endpoint, the port is configured in the UDMG Server configuration file. For example: http://udmg.stonebranch.com:8080/api |
MFT Credentials | Required | - | Credentials | UDMG Server Credentials. The Credentials definition should be as follows.
|
Source File | Required | - | Text | The name of file,
|
Target File Introduced in version 1.1.0 | Optional | - | Text | The name of file on the target system after the transfer,
|
Partner | Required | - | Dynamic Choice | Dynamically fetched list of partner names. The list is restricted to the partners that support the selected Protocol. The user can select the required partner name from a drop-down list. |
Rule | Required | - | Dynamic Choice | Dynamically fetched list of transfer rules. The list is restricted to the rules that are assigned to the selected Partner. The user can select the required rule from a drop-down list. |
Partner Account | Required | - | Text | The name of the account to use for the connection to the selected Partner. The account credentials are configured on UDMG. |
Wait for Transfer Completion Introduced in version 1.2.0 | Required | True | Boolean | By checking this field, the extension monitors the completion of the transfer on UDMG before completing its execution.
|
Polling Interval | Optional | 5 | Integer | The amount of time (in seconds) the task will wait before polling to check the transfer status. Valid values from 0. |
Max Files to Monitor Introduced in version 1.2.0 | Required | 100 | Integer | The number of files, above which no monitor will be done, apart from the successful trigger of the file transfer. In practice this means that for the actions MGET & MPUT, if a wildcard transfer results in more child transfers than the number indicated, the extension will not check the transfer status of these files, and proceed to completion with a special return code (see corresponding section in this document). |
Task Examples
PUT a file to HTTPS partner
Example for sending a file to a HTTPS
server.
The configuration on UDMG for the partner:
GET a file from an SFTP server
Example for downloading a file from an SFTP
server.
The configuration on UDMG for the partner:
Task Execution
Dynamic Commands
Command Name | Allowed Task Instance Status | Description |
---|---|---|
Pause | RUNNING | Issue the pause command on the UDMG transfer |
Resume | RUNNING | Issue the resume command on the UDMG transfer |
Cancelation
When the task instance is canceled on the Controller, a cancel command is issued on the UDMG transfer to attempt a graceful termination.
Task Output
Output Only Fields
The output fields for this Universal Extension are described below.
Field | Type | Description |
---|---|---|
Transfer ID | Text | The identifier of the transfer on UDMG |
Transfer Status | Text | The current status of the transfer Used for Task instance Extension Status. |
Transfer Start Time | Text | The date and time the transfer has started (UDMG Server time) |
Transfer End Time | Text | The date and time the transfer has ended (UDMG Server time) For a wildcard transfer, this is when the listing of matching files and the registration of the child transfers is completed. |
Transfer Step | Text | The current processing step of the transfer One of NONE, PRE TASKS, DATA, POST TASKS, ERROR TASKS or FINALIZATION |
Transfer Progress Modified in version 1.2.0 | Text | The progress of the data transfer in human readable format
|
Error Code | Text | The UDMG transfer error code. See the table below. |
Error Message | Text | The error message (if an error occurred) |
Transfer Task Number | Text | The number of the task in the processing step. Only for the Transfer Steps 'PRE TASKS', 'POST TASKS', and 'ERROR TASKS' |
Local File | Text | Local file path |
Remote File | Text | Remote file path |
Transfer UUID Introduced in version 1.1.0 | Text | The UUID of the transfer on UDMG |
These fields are also available as task instance variables, with the ops_mft_ prefix:
UDMG Transfer Error Code
Error Code | Description |
---|---|
TeOk | This is the code used by default. It indicates that there is no error |
TeUnknown | Indicates an unknown error (the details of the error can give more information). |
TeInternal | Indicates that an internal error occurred in the UDMG Server (lost connection to the database, etc.) |
TeUnimplemented | Indicates that the partner wanted to use a valid functionality, but which is not implemented by the UDMG Server. |
TeConnection | Indicates that the connection to a remote partner cannot be established. This is the most general error code. A more specific code, if available, should be used. |
TeConnectionReset | Means "connection reset by peer" |
TeUnknownRemote | Indicates that the requested remote partner is not known in the UDMG Server database. For security reasons, the external use of this error code (returning this error to a client) is discouraged for incoming connections in favor of TeBadAuthentication, at the risk of leaking information (here, the account exists, but authentication failed). |
TeExceededLimit | Indicates that the transfer cannot be processed because the defined limits have been reached (number of connections, number of transfers, etc.) |
TeBadAuthentication | Indicates a connection error caused by incorrect authentication data (user, password, certificate, etc. Details of the error can provide more information). |
TeDataTransfer | Indicates that an error occurred during data transfer. This is a general error and a more specific code, if available, should be used. |
TeIntegrity | Indicates that the integrity check has failed. |
TeFinalization | Indicates that an error occurred during the finalization of the transfer. |
TeExternalOperation | Indicates that an error occurred during the execution of pre-transfer, post-transfer, or error processing. |
TeStopped | Indicates that the transfer was stopped. |
TeCanceled | Indicates that the transfer was cancelled. |
TeFileNotFound | Indicates that the requested file was not found. |
TeForbidden | Indicates that the remote partner is not allowed to perform an action. |
TeBadSize | Indicates an error related to the size of the file (it exceeds a quota, there is not enough space left on the destination disk, etc.) |
TeShuttingDown | Indicates that the UDMG Server is shutting down |
Exit Codes
The exit codes for this Universal Extension are described below.
Exit Code | Status Classification Code | Status Classification Description | Status Description |
---|---|---|---|
0 | SUCCESS | Successful Execution | SUCCESS |
1 | FAIL | Failed Execution | FAIL: < Error Description > |
2 | AUTHENTICATION_ERROR | Bad credentials | AUTHENTICATION_ERROR: Account cannot be authenticated. |
3 | AUTHORIZATION_ERROR | Insufficient Permissions | AUTHORIZATION_ERROR: Account is not authorized to perform the requested action. |
10 | CONNECTION_ERROR | Bad connection data or connection timed out | CONNECTION_ERROR: < Error Description > |
11 | CONNECTION_ERROR | Extension specific connection error | CONNECTION_ERROR: 404 page not found |
20 | DATA_VALIDATION_ERROR | Input fields Validation Error | DATA_VALIDATION_ERROR: 400 bad request: |
21 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeUnknown |
22 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeInternal |
23 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeUnimplemented |
24 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeConnection |
25 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeConnectionReset |
26 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeUnknownRemote |
27 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeExceededLimit |
28 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeBadAuthentication |
29 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeDataTransfer |
30 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeIntegrity |
31 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeFinalization |
32 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeExternalOperation |
33 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeWarning |
34 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeStopped |
35 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeCanceled |
36 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeFileNotFound |
37 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeForbidden |
38 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeBadSize |
39 | TRANSFER_STATUS_ERROR | Transfer Execution Error | TRANSFER_STATUS_ERROR: Transfer failed with error code: TeShuttingDown |
100 | TRANSFER_STATUS_CANCEL | Unexpected Transfer Status | TRANSFER_STATUS_CANCEL: Unexpected CANCELLED status |
Extension Output
In the context of a workflow, subsequent tasks can rely on the information provided by this integration as Extension Output.
The Extension output contains attribute result
. Attribute result
contains the following sub-attributes:
Attribute | Type | Description |
---|---|---|
mft_transfer | object | The details of the main transfer record (the single transfer or the parent transfer in case of Wildcard Transfer). |
mft_transfer.transfer_id | string | The identifier of the transfer on UDMG. |
mft_transfer.status | string | The current status of the transfer One of PLANNED, RUNNING, PAUSED, INTERRUPTED, DONE, CANCELLLED or ERROR. |
mft_transfer.start | string | The date and time the transfer has started (UDMG Server time) |
mft_transfer.stop | string | The date and time the transfer has ended (UDMG Server time). For a wildcard transfer, this is when the listing of matching files and the registration of the child transfers is completed. |
mft_transfer.step | string | The current processing step of the transfer One of NONE, PRE TASKS, DATA, POST TASKS, ERROR TASKS or FINALIZATION. |
mft_transfer.transfer_progress | text | The progress of the data transfer in human readable format
|
mft_transfer.error_code | string | The error code of the transfer (if an error occurred). |
mft_transfer.error_msg | string | The error message (if an error occurred). |
mft_transfer.task_number | string | The number of the task in the processing step. Only for the steps 'PRE TASKS', 'POST TASKS', and 'ERROR TASKS'. |
mft_transfer.local_file | string | Local file path. |
mft_transfer.remote_file | string | Remote file path. |
mft_transfer.transfer_uuid Introduced in version 1.1.0 | string | The UUID of the transfer on UDMG. |
mft_transfer.transfer_info Introduced in version 1.1.0 | object | The transfer information fields. Set of metadata associated to the transfer on UDMG Server. The fields with udmg_ prefix are set internally by UDMG server. The fields with udmg_ue prefix are set by the UDMG File Transfer task. |
mft_transfer.transfer_info.udmg_ue_task_uuid Introduced in version 1.2.0 | string | The UUID of the UDMG File Transfer task. It can be used to easily correlate the UC task executions and the corresponding UDMG file transfers. |
mft_transfer.transfer_info.udmg_xfer_wildcard Introduced in version 1.2.0 | boolean | Indicator that this is a wildcard transfer. Only for MPUT and MGET. |
mft_transfer.transfer_info.udmg_xfer_childIDs Introduced in version 1.2.0 | array | A list of UDMG transfer IDs that are created in the context of this file file transfer. For each file matched the filename input, one file transfer process is created in UDMG. Only for MPUT and MGET. |
mft_transfer.transfer_info.udmg_xfer_files Introduced in version 1.2.0 | array | The list of files that matched the user input. Each of these files is transferred with a separate file transfer execution, which transfer ids can be found in “udmg_xfer_childIDs“ list. Only for MPUT and MGET. |
mft_transfer_files_summary Introduced in version 1.2.0 | object | Summary counters |
mft_transfer_files_summary.done | number | Number of file transfers with success. |
mft_transfer_files_summary.error | number | Number of file transfers in error state. |
mft_transfer_files_summary.cancelled | number | Number of file transfers that are cancelled. |
errors Introduced in version 1.2.0 | array | List of errors for the child transfers. |
An example of the Extension Output for an single download from SFTP
is presented below.
An example of the Extension Output for an wildcard download from SFTP
with 3 matching files is presented below.
STDOUT and STDERR
STDERR
provides additional information to User about the task instance execution. Verbosity is increased with the Task Log Level, The Debug level can lead to extensive details and is only recommended for troubleshooting.
The populated content can be changed in future versions of this extension without notice. Backward compatibility is not guaranteed.
STDOUT
provides a summary report of the child file transfers for a wildcard transfer (only for MPUT and MGET, and only if Wait For Transfer Completion is set). Backward compatibility is not guaranteed.
Field | Description |
---|---|
id | The identifier of the transfer on UDMG |
status | The current status of the transfer |
localFilepath | The path of the file on the local disk |
start | The date and time the transfer has started (UDMG Server time) |
stop | The date and time the transfer has ended (UDMG Server time) |
filesize | The size of file in byte (or -1 when unknown) |
progress | The progress of the data transfer in bytes |
remoteID | The UUID of the transfer on UDMG |
errorCode | The UDMG transfer error code. |
errorMessage | The error message (if an error occurred) |
step | The current processing step of the transfer One of NONE, PRE TASKS, DATA, POST TASKS, ERROR TASKS or FINALIZATION |
taskNumber | The number of the task in the processing step. Only for the Transfer Steps 'PRE TASKS', 'POST TASKS', and 'ERROR TASKS' |
remoteFilepath | The path to the file on the remote partner |
rule | The name of the trasnfer rule |
isSend | Indicates whether the transfer is a send (true) or a receive (false) |
requested | The name of the server/partner that requested the transfer |
requester | The name of the account that requested the transfer |
An example of the STDOUT for an wildcard download from SFTP
with 3 matching files is presented below.
Integration Modifications
Modifications applied by users or customers, before or after import, might affect the supportability of this integration. The following modifications are discouraged to retain the support level as applied for this integration.
- Python code modifications should not be done.
- Template Modifications
General Section
- "Name", "Extension", "Variable Prefix", "Icon" should not be changed.
Universal Template Details Section
- "Template Type", "Agent Type", "Send Extension Variables", "Always Cancel on Force Finish" should not be changed.
Result Processing Defaults Section
- Success and Failure Exit codes should not be changed.
- Success and Failure Output processing should not be changed.
Fields Restriction Section
The setup of the template does not impose any restrictions, However with respect to "Exit Code Processing Fields" section.- Success/Failure exit codes need to be respected.
- In principle, as STDERR and STDOUT outputs can change in follow-up releases of this integration, they should not be considered as a reliable source for determining success or failure of a task.
Users and customers are encouraged to report defects, or feature requests at Stonebranch Support Desk.
Document References
This document references the following documents.
Document Link | Description |
---|---|
Universal Templates | User documentation for creating, working with and understanding Universal Templates and Integrations. |
Universal Tasks | User documentation for creating Universal Tasks in the Universal Controller user interface. |
Credentials | User documentation for creating and working with credentials. |
Resolvable Credentials Permitted Property | User documentation for Resolvable Credentials Permitted Property. |
Changelog
ue-mft-transfer-1.2.0 (2023-05-09)
Deprecations and Breaking Changes
Breaking Change
: Transfer Progress output only field is changed to text value (#32195)
Enhancements
Add
: Support Wildcard Transfer (#32198, #32195, #32201)- New actions "MPUT" and "MGET" for wildcard transfers:
- support for wildcard in source file parameter,
- creation of wildcard transfer on UDMG,
- monitoring of child transfers on UDMG,
- tabular report on STDOUT with final child transfer status,
- short summary report in EXTENSION output,
- PAUSE, RESUME, and CANCEL commands are propagated to child transfers
- "Transfer Progress" output field is redefined to show human readable value
- size in KB, MB, or GB for single transfer,
- "x out of y files" for wildcard transfer
- "Max files to monitor" parameter: limit the number of child transfers that are monitored by the task instances, 100 by default.
- "Wait for transfer completion" parameter: task instance only returns when the transfer is completed (with DONE or CANCELLED status) on UDMG.
For wildcard transfers, only returns when all child transfers are completed. Activated by default.
- New actions "MPUT" and "MGET" for wildcard transfers:
Add
: Refresh task instance output with re-run (#32463)Add
: Keep UC task instance ID in UDMG file transfer metadata field udmg_ue_task_uuid (#30931)
ue-mft-transfer-1.1.0 (2023-01-31)
Deprecations and Breaking Changes
Breaking Change
: minimum universal agent version set to 7.3
Enhancements
Add
: set target filename (rename file) (#31174)Add
: transfer status as UC 7.3 extension status field. (#31041)Add
: retry failed transfer with Task Re-run. (#29599)Add
: show UUID. (#30733)Add
: show transfer info fields. (#30930)Add
: support for PeSIT and PeSIT-TLS protocols (#31051)
Fixes
Fix
: optimize number of API calls. (#29247)
ue-mft-transfer-1.0.0 (2022-06-30)
- Initial version