Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Rclone needs to be installed on the same server where the Universal Agent is installed. Rclone binary should be stored on a location where Universal Agent has access and permissions to execute it.

Note

This Universal Extension has been tested with Rclone v1.58.1. It should be working with later Rclone versions, as long as these are backwards compatible.


Network and Connectivity Requirements

...

  • Amazon S3
  • Google Cloud Storage
  • Microsoft OneDrive Business
  • Local file system (Linux, Windows)
  • HTTP(S) URL
Note

This integration should work properly against other Storage Systems, as long as the integration interface (see chapter Configure Universal Task) and functionalities listed in this document fulfil the needs of the Storage System.

For functionalities required for specific Storage Systems, users and customers are encouraged to open a Feature Request in our Customer Support Portal.


Import Universal Template

...

  1. The configuration file is following the INI format, and contains all required parameters and credentials to connect to a Storage System. All configuration options per Storage System can be found in the respective "Config Rclone documentation" here.

  2. It is advised that account credentials, tokens, or any other essential information to be passed as an encrypted Resolvable Credential field. To set a value with the respective Credential field, please make sure to use the appropriate Credential function inside the configuration file.

    Please
    Note
    that

    'Runtime Password' Credentials field, supports values with up 512 characters.


  3. Hard coded values can be provided in the script file, as well. However, it is not advised for security reasons.

  4. It is recommended to configure a storage system section, one to one with the Credential fields in this Universal Extension (please refer to the Input Fields section below).

  5. For local file system Storages, no credentials are required in the configuration file. The selected action will be executed with the Credentials that are provided in the "Agent Details" of the Universal Task. If no Credentials are provided for the used Agent, then the same user that runs ubroker daemon will be used.

  6. Script Type should be Data, and option Resolve UAC Variables should be enabled.


The indicative Configuration File below, contains the basic connection parameters (flags):

Note

Utilize the built-in Credential Functions, in order to properly resolve the associated Universal Task Credentials.


[ue_aws_s3]
type = s3
provider = AWS
access_key_id = ${_credentialUser('${ops_ue_cloud_ft_storage_credentials}')}
secret_access_key = ${_credentialPwd('${ops_ue_cloud_ft_storage_credentials}')}
region = us-east-2

[ue_aws_s3_source]
type = s3
provider = AWS
access_key_id = ${_credentialUser('${ops_ue_cloud_ft_source_credentials}')}
secret_access_key = ${_credentialPwd('${ops_ue_cloud_ft_source_credentials}')}
region = us-east-2
acl = bucket-owner-full-control
role_arn = arn:aws:iam::<account_id>:role/<role policy>

[ue_aws_s3_target]
type = s3
provider = AWS
env_auth = false
access_key_id = ${_credentialUser('${ops_ue_cloud_ft_target_credentials}')}
secret_access_key = ${_credentialPwd('${ops_ue_cloud_ft_target_credentials}')}
region = us-east-2

[google_cloud_storage]
type = google cloud storage
service_account_file = ${_credentialPwd('${ops_ue_cloud_ft_storage_credentials}')}
object_acl = bucketOwnerFullControl
project_number = johnprojectno
location = europe-west3

[google_cloud_storage_source]
type = google cloud storage
service_account_file = ${_credentialPwd('${ops_ue_cloud_ft_source_credentials}')}
object_acl = bucketOwnerFullControl
project_number = johnprojectno
location = europe-west3

[google_cloud_storage_target]
type = google cloud storage
service_account_file = ${_credentialPwd('${ops_ue_cloud_ft_target_credentials}')}
object_acl = bucketOwnerFullControl
project_number = johnprojectno
location = europe-west3
 
[one_drive]
type = onedrive
token = ${_credentialToken('${ops_ue_cloud_ft_storage_credentials}')}
drive_id = ${_credentialUser('${ops_ue_cloud_ft_storage_credentials}')}
drive_type = business
auth_url = https://login.microsoftonline.com/<tenant id>/oauth2/v2.0/authorize
token_url = https://login.microsoftonline.com/<tenant id>/oauth2/v2.0/token

 
[one_drive_source]
type = onedrive
token = ${_credentialToken('${ops_ue_cloud_ft_source_credentials}')}
drive_id = ${_credentialUser('${ops_ue_cloud_ft_source_credentials}')}
drive_type = business
 
[one_drive_target]
type = onedrive
token = ${_credentialToken('${ops_ue_cloud_ft_target_credentials}')}
drive_id = ${_credentialUser('${ops_ue_cloud_ft_target_credentials}')}
drive_type = business
 
[linux_source]
type = local
 
[linux_target]
type = local
 
[windows_source]
type = local
 
[windows_target]
type = local

...

Document LinkDescription
Universal TemplatesUser documentation for creating Universal Templates in the Universal Controller user interface.
Universal TasksUser documentation for creating Universal Tasks in the Universal Controller user interface.
RcloneRclone official documentation.
Rclone InstallInstallation instructions.
Rclone DownloadsDownload links for Linux, Windows, MacOS.
Rclone Storage Systems configurationConfiguration file details.
Rclone Exit CodesRclone exit codes list.

Changelog

ue-cloud-ft-1.0.0 (2022-09-

...

02)

Added: Initial version (#28787)