...
This Universal Extension is an interface to Rclone that provides the capability to monitor files/directories from across different cloud storages, as well as local or distributed file systems. Additionally and upon successful monitor, this extension publishes Local Universal Events. To properly handle these events and take action on them, this Universal Extension optionally can be attached as a Publisher to a Universal Monitor Task (see more details in section Cloud File Monitor Events).
Rclone, is the open source command line program, that is utilized to accomplish all the actions supported in this Extension.
...
Template Name | Extension Name | Extension Version |
---|---|---|
Cloud File Monitor | ue-cloud-fm | 1.0.0 |
Refer to Changelog RClone: Cloud File Monitor for version history information.
...
Field | Input type | Default value | Type | Description |
---|---|---|---|---|
Action | Required | Monitor On Create | Choice | Action to be performed. Available actions:
|
Trigger On Existence | Optional | False | Boolean | When this flag is enabled and objects already exist on the given Storage Filepath , an event will be published. Otherwise, Universal Task will keep monitoring for new objects creation. Optional when Action Monitor On Create is selected. |
Configuration File | Required | - | Script | Rclone Configuration file, that contains all required parameters to connect to the Storage System. |
Storage System | Optional | - | Dynamic Choice Field | Remote or Local storage to execute the selected Action. Execute the Dynamic Choice Field and retrieve all the available Storages from Configuration File. This field is included as an Event Attribute. |
Storage Credentials | Optional | - | Credential | Credentials needed to connect to the selected Storage System. For Storage System of type local , the standard Agent's Credentials field should be provided. |
Storage Filepath | Optional | - | Text | Path to monitor objects for the selected Action. This field is included as an Event Attribute. |
Update Credentials | Optional | False | Boolean | This field needs to be used when Rclone executable updates Fields on its Configuration File during execution, that need to be stored as Credential Fields on the Universal Controller. For example: when OneDrive Storage type is used. In this case, the existing configured token is exchanged by Rclone with a new one to establish the connection. This new token should be stored on Universal Controller for the next task execution. Please read Rclone limitations on OneDrive token refresh here. This field is used in conjunction with fields Controller URL, Controller Credentials, Refresh Storage Credentials. |
Controller URL | Optional | - | Text | Controller URL where Storage Credentials are stored. Required when Update Credentials is checked. |
Controller Credentials | Optional | - | Credentials | Controller user's credentials, used for logging and updating Storage Credentials will be updated. The Credentials definition should be as follows:
Required when Update Credentials is checked. |
Refresh Storage Credentials | Optional | - | Array | This Array field is available when Update Credentials is set to True and used as a mapping table that indicates which field from the Configuration File of Rclone is used to update which Credential entry field on Universal Controller. Populate this array with the following format:
Note: Refresh of Storage Credentials is performed via Controller's REST API. See Network and Connectivity Requirements |
Monitor Time Interval | Optional | 300 | Integer | Interval period in seconds used to monitor the given Storage Filepath. |
Use Filter | Optional | -- None -- | Choice | The filter type that is applied on the Action. Available options:
|
Filter | Optional | - | Text | Filter based on Rclone filtering rules that is applied on the Action. Required when Use Filter is checked. |
Files Only | Optional | False | Boolean | When enabled, directories will be ignored and only files will be monitored. |
Use Storage System Modified Time | Optional | True | Boolean | When enabled, the time of the Storage System will be observed as the object's modification time, instead of the original object's modified time stored as metadata. Note: Having this option enabled, increases performance. |
Recursion Depth | Optional | 1 | Integer | Recursion depth that is applied. Default value 1 means that no recursion will be applied. |
Additional Options | Optional | - | Text | Space separated Rclone options that applied to the selected Action. |
Correlation ID | Optional | - | Text | Use this field optionally to pass extra custom information to the published event (for example: provide the parent Workflow task instance ID). This field is included as an Event Attribute. |
Anchor | ||||
---|---|---|---|---|
|
This Universal Extension publishes Universal Events when objects are monitored succesfully, based on the selected Action. The suggested implementation to accomplish this behavior is to associate a Cloud File Monitor Task(event publisher) with a Universal Monitor Task(event consumer).
...