/
UDMG File Transfer

UDMG File Transfer

Feb 02, 2024



Disclaimer

Your use of this download is governed by Stonebranch’s Terms of Use, which are available at https://www.stonebranch.com/integration-hub/Terms-and-Privacy/Terms-of-Use/

Overview

UDMG (Universal Data Mover Gateway) is a business-to-business MFT (Managed File Transfer) Solution. It provides the capabilities to securely manage File transfers with external business partners (i.e. Customers or Suppliers), and between any endpoint in today’s Hybrid-IT infrastructure, on-premises, or in the cloud (public and or private).

This Universal Extension provides the capability to perform file transfer where UDMG is the client and upload or download files to remote partners.

Version Information

Template Name

Extension Name

Extension Version

Template Name

Extension Name

Extension Version

UDMG File Transfer

ue-mft-transfer

1.3.0

Refer to Changelog for version history information.

Software Requirements

This integration requires a Universal Agent and a Python runtime to execute the Universal Task.

Software Requirements for Universal Template and Universal Task

Requires Python 3.7.0 or higher. Tested with the Universal Agent bundled Python distribution.

UDMG Server 1.2.1.0 or higher.

Software Requirements for Universal Agent

Both Windows and Linux agents are supported.

  • Universal Agent for Windows x64 Version 7.3.0.0 and later with python options installed.

  • Universal Agent for Linux Version 7.3.0.0 and later with python options installed.

Software Requirements for Universal Controller

Universal Controller Version 7.3.0.0 and later.

Network and Connectivity Requirements

Universal Agent requires connectivity towards the UDMG Server REST API.

Key Features

This Universal Extension provides the following key features:

  • Actions

    • GET: trigger a File Transfer on UDMG to download a file from a remote partner.

    • PUT: trigger a File Transfer on UDMG to upload a file to a remote partner.

    • MGET:trigger a Wildcard File Transfer on UDMG to download several files from a remote partner.

    • MPUT: trigger a Wildcard File Transfer on UDMG to upload several files to a remote partner. 

    • LIST: trigger a Wildcard List File Transfer on UDMG to list the files matching a wildcard pattern on a remote SFTP partner. 

  • File Transfer Control

    • Capability to pause a partial transfer.

    • Capability to resume a paused transfer.

    • Capability to cancel a transfer (from planned, running, paused, or error status).

    • Capability to retry a failed transfer.

    • Synchronous mode: the task monitors the UDMG transfer and terminates when the transfer is completed (either with success or error).

    • Asynchronous mode: the task registers the UDMG transfer and terminates when the transfer request is acknowledged on UDMG.

  • Input/Output

    • Capability for the UAC task author, to retrieve the list of partners from UDMG for a given protocol.

    • Capability for the UAC task author, to retrieve the list of transfer rules from UDMG for a given partner.

    • Monitor of the transfer with periodic polling from UDMG and display to output-only fields and extension output.

  • Other

    • Support for SFTP, HTTPS, PeSIT, and PeSIT-TLS file transfer protocols.

Import Universal Template

To use the Universal Template, you first must perform the following steps.

  1. This Universal Task requires the Resolvable Credentials feature. Check that the Resolvable Credentials Permitted system property has been set to true.

  2. To import the Universal Template into your Controller, follow the instructions here.

  3. When the files have been imported successfully, refresh the Universal Templates list; the Universal Template will appear on the list.



Modifications of this integration, applied by users or customers, before or after import, might affect the supportability of this integration. For more information refer to IntegrationModifications.

Configure Universal Task

For a new Universal Task, create a new task, and enter the required input fields.

Input Fields

The input fields for this Universal Extension are described below.

Field

Input type

Default value

Type

Description

Field

Input type

Default value

Type

Description

Action

Required

PUT

Choice

Action performed upon the task execution.
Available actions:

  • GET

  • PUT

  • MGET

  • MPUT

  • LIST

Protocol

Required

SFTP

Choice

The transfer protocol.
Available values:

  • SFTP

  • HTTP

  • HTTPS

  • PeSIT

  • PeSIT-TLS

Note

The protocol serves as a filter for the Partner list. The actual protocol for the file transfer is the one that is configured on UDMG for the selected Partner

MFT Server

Required

-

Text

UDMG Server URL. This is the URL for the REST API endpoint, the port is configured in the UDMG Server configuration file.
For example: http://udmg.stonebranch.com:8080/api

MFT Credentials

Required

-

Credentials

UDMG Server Credentials. The Credentials definition should be as follows.

  • User Name as "Runtime User".

  • User Password as "Runtime Password".

The user must have read permissions for rules and partners, read/write permission for transfers

Source File

Required

-

Text

The name of the source file, or the search pattern:

  • PUT: local file

  • GET: remote file

  • MPUT: wildcard pattern for local files

  • MGET: wildcard pattern for remote files

  • LIST: wildcard pattern for remote files

Target File

Introduced in version 1.1.0

Optional

-

Text

The name of file on the target system after the transfer,

  • PUT: remote file

  • GET: local file

  • MPUT/MGET/LIST: not used, the field is not displayed.

Partner

Required

-

Dynamic Choice

Dynamically fetched list of partner names.
The list is restricted to the partners that support the selected Protocol. The user can select the required partner name from a drop-down list.

Rule

Required

-

Dynamic Choice

Dynamically fetched list of transfer rules.
The list is restricted to the rules that are assigned to the selected Partner. The user can select the required rule from a drop-down list.

Partner Account

Required

-

Text

The name of the account to use for the connection to the selected Partner.
The account credentials are configured on UDMG.

Wait for Transfer Completion

Introduced in version 1.2.0

Required

True

Boolean

By checking this field, the extension monitors the completion of the transfer on UDMG before completing its execution.

  • Actions PUT/GET: Wait for the single transferred to be completed.

  • Actions MPUT/MGET/LIST: Wait for the child file transfers to be completed or not. The extension will always wait for the parent process to be completed successfully, regardless of this configuration attribute.

Polling Interval

Optional

5

Integer

The amount of time (in seconds) the task will wait before polling to check the transfer status.
Valid values from 1.

Max Files to Monitor

Introduced in version 1.2.0

Required

100

Integer

The number of files, above which no monitor will be done, apart from the successful trigger of the file transfer.

In practice this means that for the actions MGET & MPUT, if a wildcard transfer results in more child transfers than the number indicated, the extension will not check the transfer status of these files, and proceed to completion with a special return code (see corresponding section in this document).

Task Examples

PUT a file to HTTPS partner

Example for sending a file to a HTTPS server.



The configuration on UDMG for the partner:

GET a file from an SFTP server

Example for downloading a file from an SFTP server.

The configuration on UDMG for the partner:

MGET files from a remote SFTP server

Example for downloading multiple files from an SFTP server.

LIST files on a remote SFTP server

Example for listing files from an SFTP server.

Task Execution

Dynamic Commands

Command Name

Allowed Task Instance Status

Description

Command Name

Allowed Task Instance Status

Description

Pause

RUNNING

Issue the pause command on the UDMG transfer.

For a wildcard transfer (MPUT or MGET), the pause command is issued for all child transfers on UDMG. Only the eligible children, in status PLANNED or RUNNING, are considered. Failure on one transfer does not block the execution for the other transfers.

Resume

RUNNING

Issue the resume command on the UDMG transfer.

For a wildcard transfer (MPUT or MGET), the resume command is issued for all child transfers on UDMG. Only the eligible children, in status INTERRUPTED or PAUSED, are considered. Failure on one transfer does not block the execution for the other transfers.

Cancelation

When the task instance is canceled on the Controller, a cancel command is issued on the UDMG transfer to attempt a graceful termination.

  • If the transfer has no children (PUT, GET, LIST action):

    • Fetch the UDMG transfer status and update the output-only fields.

    • Proceed to cancel the UDMG transfer only when its status is one of PLANNED, RUNNING, INTERRUPTED, PAUSED, or ERROR.

    • Update output only fields when the operation is completed.

  • If the transfer has children (MPUT, MGET action): cancel the execution of all of the eligible child transfers on UDMG ( in status PLANNED, RUNNING, INTERRUPTED, PAUSED, ERROR). Failure on one transfer does not block the execution for the other transfers.

Re-Run

When the task instance is re-run on the Controller, a resume command is issued on the UDMG transfer for the file transfers in status ERROR.

  • If the transfer has no children (PUT, GET, LIST action):

    • Fetch the UDMG transfer status and update the output-only fields.

    • Proceed to resume the UDMG transfer only when its status is ERROR, otherwise

    • Continute the monitoring until the UDMG transfer is completed.

  • If the transfer has children (MPUT, MGET action): resume the execution of all of the eligible child transfers on UDMG (in status  ERROR). Failure on one transfer does not block the execution for the other transfers.

Task Output

Output Only Fields

The output fields for this Universal Extension are described below.

Field

Type

Description

Field

Type

Description

Transfer ID

Text

The identifier of the transfer on UDMG

Transfer Status

Text

The current status of the transfer
One of PLANNED, RUNNING, PAUSED, INTERRUPTED, DONE, CANCELLED or ERROR

Used for Task instance Extension Status.

Transfer Start Time

Text

The date and time the transfer has started (UDMG Server time)

Transfer End Time

Text

The date and time the transfer has ended (UDMG Server time)

For a wildcard transfer, this is when the listing of matching files and the registration of the child transfers is completed.

Transfer Step

Text

The current processing step of the transfer
One of NONE, PRE TASKS, DATA, POST TASKS, ERROR TASKS or FINALIZATION

Transfer Progress

Modified in version 1.2.0

Text

The progress of the data transfer in human readable format

  • size in KB, MB, or GB for a Single Transfer,

  • "x out of y files" for a Wildcard Transfer

Error Code

Text

The UDMG transfer error code.

See the table below.

Error Message

Text

The error message (if an error occurred)

Transfer Task Number

Text

The number of the task in the processing step.
Only for the Transfer Steps 'PRE TASKS', 'POST TASKS', and 'ERROR TASKS'

Local File

Text

Local file path

Remote File

Text

Remote file path

Transfer UUID

Introduced in version 1.1.0

Text

The UUID of the transfer on UDMG

These fields are also available as task instance variables, with the ops_mft_ prefix:

UDMG Transfer Error Code

Error Code

Description

Error Code

Description

TeOk

This is the code used by default. It indicates that there is no error

TeUnknown

Indicates an unknown error (the details of the error can give more information).

TeInternal

Indicates that an internal error occurred in the UDMG Server (lost connection to the database, etc.)

TeUnimplemented

Indicates that the partner wanted to use a valid functionality, but which is not implemented by the UDMG Server.

TeConnection

Indicates that the connection to a remote partner cannot be established. This is the most general error code. A more specific code, if available, should be used.

TeConnectionReset

Means "connection reset by peer"

TeUnknownRemote

Indicates that the requested remote partner is not known in the UDMG Server database. For security reasons, the external use of this error code (returning this error to a client) is discouraged for incoming connections in favor of TeBadAuthentication, at the risk of leaking information (here, the account exists, but authentication failed).

TeExceededLimit

Indicates that the transfer cannot be processed because the defined limits have been reached (number of connections, number of transfers, etc.)

TeBadAuthentication

Indicates a connection error caused by incorrect authentication data (user, password, certificate, etc. Details of the error can provide more information).

TeDataTransfer

Indicates that an error occurred during data transfer. This is a general error and a more specific code, if available, should be used.

TeIntegrity

Indicates that the integrity check has failed.

TeFinalization

Indicates that an error occurred during the finalization of the transfer.

TeExternalOperation

Indicates that an error occurred during the execution of pre-transfer, post-transfer, or error processing.

TeStopped

Indicates that the transfer was stopped.

TeCancelled

Indicates that the transfer was cancelled.

TeFileNotFound

Indicates that the requested file was not found.

TeForbidden

Indicates that the remote partner is not allowed to perform an action.

TeBadSize

Indicates an error related to the size of the file (it exceeds a quota, there is not enough space left on the destination disk, etc.)

TeShuttingDown

Indicates that the UDMG Server is shutting down

Exit Codes

The exit codes for this Universal Extension are described below.

Exit Code

Status Classification Code

Status Classification Description

Status Description

Exit Code

Status Classification Code

Status Classification Description

Status Description

0

SUCCESS

Successful Execution

SUCCESS

1

FAIL

Failed Execution

FAIL: < Error Description >

2

AUTHENTICATION_ERROR

Bad credentials