Disclaimer

Your use of this download is governed by Stonebranch’s Terms of Use, which are available at Stonebranch Integration Hub - Terms of Use.

Version Information

Template Name	Extension Name	Version	Status
Cloud Data Transfer	ue-cloud-dt	4 (Current 4.0.2)	Fixes and new features are introduced.
Cloud Data Transfer	ue-cloud-dt	3	Hot Fixes Only (Until UAC 7.3 is End of Support)
Cloud Data Transfer	ue-cloud-dt	1 & 2	End of Support

Refer to Changelog for version history information.

Version 4.0.0 is a major release and introduces breaking changes that might affect some users depending on their setup. Administrators are strongly advised to refer to Changelog for more information on the changes introduced in this release. Refer also to Migration Guide to apply changes in existing task definitions.

Overview

This integration provides the capability to perform data transfers between cloud-based storage services and local or distributed file systems. It also provides data storage management capabilities like listing, creating, or deleting data storage objects.

Key Features

This integration is equipped with the following key features.

Feature	Description
Data management and synchronization	Copy, move, and synchronize data between two storages. Copy a URL's content to a cloud or local destination without saving it in temporary storage. List data on a storage service, including listing with details or in JSON format for machine parsing. Create objects on a storage. Delete objects from a storage.
Efficient and secure data transfers	Fast transfers for objects stored in the same region. Preserve timestamps and checksums verification. Transfers can optionally support encryption, caching, compression, and chunking. Dynamic token refresh for OneDrive cloud storage, observing the OAuth2 authorization and sign-in flow. Support for dry runs. Allows users to execute a Universal Task without making any permanent changes to the target storage. Advanced filtering capability for files or objects to be listed or transferred.
Complementary capabilities	Additional customized options. Option to mark the Universal Task as Failed when no files have been transferred. Progress of the selected Action is visible, during Universal Task Instance execution. Text or JSON formatted output.
Observability	Rich observability capabilities (starting from UAC 7.5.0.0)

Requirements

This integration requires a Universal Agent, a Python runtime to execute the Universal Task.

Area	Details
Python Version	Requires Python 3.7 or 3.11. Tested with Python 3.7.6 and Python 3.11.6.
Universal Agent Compatibility	Compatible with Universal Agent for Windows x64 and version >= 7.4.0.0. Compatible with Universal Agent for Linux and version >= 7.4.0.0.
Universal Controller Compatibility	Universal Controller Version >= 7.4.0.0.
Network and Connectivity	Universal Agent should be able to establish a connection with: The remote storage providers used at the Universal Task Definition. The Universal Controller host. This is required only in case Refresh Storage Credentials field is used

Supported Actions

This integration supports multiple actions which can be grouped into the following categories:

Category	Actions	Required Input Fields
Single storage actions	List Objects List Directories List Objects With Details List Objects In JSON Format Create Object Delete Object Delete Empty Directory	Storage System Storage Credentials Filepath
Two storage actions	Copy Objects Copy And Rename Copy From URL Move Objects Move And Rename Synchronize	Source Storage Source Credentials Source Filepath Target Storage Target Credentials Target Filepath

Action Type: Single Storage Action

Single storage actions involve listing and managing existing data objects in a local or remote storage.

Configuration Examples

User Scenario:

List Directories action, using a filter and with output log in Text format.

User Scenario:

Create Object action and with output as JSON format.

Action Output

Output Type

Description

Examples

EXTENSION

The extension output provides the following information:

exit_code, status_description: General info regarding the task execution. For more information, users can refer to the Exit Codes section.
invocation.fields: The task configuration used for this task execution.
result.cmd: The executed command line.
result.rc: The return code from the executed command line.

Successful Execution

{
	"exit_code": 0,
	"status_description": "Task executed successfully",
	"invocation": {
		"extension": "ue-cloud-dt",
		"version": "4.0.0",
		"fields": {
			"action": "List Directories",
             ...
		}
	},
	"changed": false,
	"result": {
		"cmd": "rclone lsd ue_aws_s3:john-drive/--config=C:\\DATA\\tmp\\54cd05b2-1528-46d2-8f9c- 8bc09dfdbf4.conf --log-level=DEBUG --max-depth=4 --use-json-log --exclude ue_execution/compiled",
		"rc": 0
	}
}

Failed Execution

{
&nb

Integrations

Inter-Cloud Data Transfer