Panel | |
---|---|
|
Disclaimer
...
The following file transfer commands are supported:
Upload a file(s) to an S3 bucket
Download of file(s) from an S3 bucket
Transfer files between S3 buckets
List objects in an S3 bucket
Delete object(s) in an S3 bucket
List S3 bucket names
Create an S3 bucket
Monitor for a key(s) in a bucket.
File Transfer can be trigger by a third-party application using the Universal Automation Center RESTfull Webservice API: REST API.
Universal Task for AWS S3 can be integrated into any existing scheduling workflow in the same way as any standard Linux or Windows Task type.
Security is ensured by using the HTTPS protocol with support for an optional Proxy Server.
AWS IAM Role Based Access ( RBCA ) is supported.
AWS canned ACLs are supported; for example, to grant full access to the bucket owner.
No Universal Agent needs to be installed on the AWS Cloud – the communication goes via HTTPS.
Version Information
Template Name | Version |
---|---|
AWS-S3 | 1.5.25 |
Refer to Changelog for version history information.
Software Requirements
Software Requirements Universal Agents and Universal Controller
...
Name | Description |
Upload a file(s) to a bucket (Copy or Move) |
|
Download of file(s) from a bucket (Copy or Move) |
|
Copy object to a bucket |
|
List objects in a bucket |
|
Delete object(s) in a bucket |
|
List buckets | List all bucket in an AWS S3 account |
Create an S3 bucket | Create a new S3 bucket |
Delete an S3 bucket | Delete an S3 bucket |
Monitor Key | Monitor if a certain key(s) exists in an S3 bucket. Unix filename pattern matching support; for example, wildcard wild card support “*” to narrow down the keys to be monitored. |
Proxy Server Connection | A proxy server connection with or without basic authentication can be configured |
Integration into 3rd Party Applications | An AWS S3 file transfer can be triggered via the Universal Automation Center RESTfull Webservice API within an application. |
Self-Service through Web-Client | The S3 Task can be fully configured, monitored and updated via the Universal Controller Web-GUI |
...
This Universal Task requires the Resolvable Credentials feature. Check that the Resolvable the Resolvable Credentials Permitted system property has been set to true.
In the Universal Controller UI, select Administration >Configuration > Universal Templates to display the current list of Universal Templates.
Click Import Template.
Select the template ZIP file and Import.
...
Field | Description |
Action | Upload-file action |
AWS_ACCESS_KEY_ID | AWS Account credentials Access Key |
AWS_DEFAULT_REGION | AWS Account credentials Region |
AWS_SECRET_ACCESS_KEY | AWS Account credentials Secret Access Key |
Useproxy ( default is NO ) | [NO | YES] If set to YES, the fields to set-up the proxy server connections are displayed:
|
Loglevel ( default is INFO ) | Universal Task logging settings [DEBUG | INFO| WARNING | ERROR | CRITICAL] |
Role Based Access | [NO | YES] If set to YES, the fields to set-up the IAM Role Based Access Connection are displayed:
STS: AWS Security Token Service is used to create and provide trusted users with temporary security credentials that can control access to your AWS resources. |
ACL | bucket-owner-full-control | private | public-read | public-read-write | aws-exec-read | authenticated-read | bucket-owner-read | log-delivery-write AWS canned ACLs; for details, refer to: The default ACL bucket-owner-full-control grant full access to the bucket owner. |
Endpoint URL | Only used in case of a MinIO server; connection Endpoint URL for the MinIO storage |
Sourcefile | Source file(s) to be uploaded to a bucket or specific folder in a bucket. Unix filename pattern are supported to upload a selection of files. Supported wildcards are:
|
Operation | [copy | move] In case of a “move” the source file(s) is/are deleted after the upload to the bucket. |
Prefix | A folder in a bucket is called prefix in AWS. In the field Prefix only the name of the folder needs to be provided no “/” after the folder name. |
Upload Write Options | Upload Write Options [Replace existing Object | Do not overwrite existing Object | Timestamp]
|
...
Field | Description |
Action | Delete-objects action |
AWS_ACCESS_KEY_ID | AWS Account credentials Access Key |
AWS_DEFAULT_REGION | AWS Account credentials Region |
AWS_SECRET_ACCESS_KEY | AWS Account credentials Secret Access Key |
Useproxy ( default is NO ) | [NO | YES] If set to YES, the fields to set-up the proxy server connections are displayed:
|
Loglevel ( default is INFO ) | Universal Task logging settings [DEBUG | INFO| WARNING | ERROR | CRITICAL] |
Role Based Access | [NO | YES] If set to YES, the fields to set-up the IAM Role Based Access Connection are displayed:
STS: AWS Security Token Service is used to create and provide trusted users with temporary security credentials that can control access to your AWS resources. |
Endpoint URL | Only used in case of a MinIO server; connection Endpoint URL for the MinIO storage system |
Bucket | AWS bucket name |
S3key | Key to be deleted in AWS Note: Due to security reasons wildcard wild card is only support, if at least one character is provided e.g. t* would delete all files starting with at “t”. |
...
Field | Description | ||||||||||
Action | Copy-object-to-bucket action | ||||||||||
AWS_ACCESS_KEY_ID | AWS Account credentials Access Key | ||||||||||
AWS_DEFAULT_REGION | AWS Account credentials Region | ||||||||||
AWS_SECRET_ACCESS_KEY | AWS Account credentials Secret Access Key | ||||||||||
Useproxy ( default is NO ) | [NO | YES] If set to YES, the fields to set-up the proxy server connections are displayed:
| ||||||||||
Loglevel ( default is INFO ) | Universal Task logging settings [DEBUG | INFO| WARNING | ERROR | CRITICAL] | ||||||||||
Role Based Access | [NO | YES] If set to YES, the fields to set-up the IAM Role Based Access Connection are displayed:
STS: AWS Security Token Service is used to create and provide trusted users with temporary security credentials that can control access to your AWS resources. | ||||||||||
ACL | bucket-owner-full-control | private | public-read | public-read-write | aws-exec-read | authenticated-read | bucket-owner-read | log-delivery-write AWS canned ACLs; for details, refer to: The default ACL bucket-owner-full-control grant full access to the bucket owner. | ||||||||||
Endpoint URL | Only used in case of a MinIO server; connection Endpoint URL for the MinIO storage system | ||||||||||
Bucket | AWS bucket name | ||||||||||
Target Bucket | target Bucket, where the object(s) will be copied to | ||||||||||
S3key | File to be copied from one bucket to another | ||||||||||
TargetS3Key | Target file name. The TargetS3key supports pattern match. For example: Bucket: stonebranchpm Target Bucket: stonebranchpm1 S3key: dir1/report* ; Target s3key: dir2/* This copies all files from bucket stonebranchpm in the directory dir1 starting with the file name report to the directory dir2/ in the bucket stonebranchpm1.
| ||||||||||
Upload Write Options | Upload Write Options [Replace existing Object | Do not overwrite existing Object | Timestamp]
|
...
Example for Amazon S3 Universal Tasks - Delete Bucket
Monitor Key - Action
This Action is used to monitor for a key(s) in a bucket
Field | Description |
Action | monitor-key action |
AWS_ACCESS_KEY_ID | AWS Account credentials Access Key |
AWS_DEFAULT_REGION | AWS Account credentials Region |
AWS_SECRET_ACCESS_KEY | AWS Account credentials Secret Access Key |
Useproxy ( default is NO ) | [NO | YES] If set to YES, the fields to set-up the proxy server connections are displayed:
|
Loglevel ( default is INFO ) | Universal Task logging settings [DEBUG | INFO| WARNING | ERROR | CRITICAL] |
Role Based Access | [NO | YES] If set to YES, the fields to set-up the IAM Role Based Access Connection are displayed:
STS: AWS Security Token Service is used to create and provide trusted users with temporary security credentials that can control access to your AWS resources. |
Endpoint URL | Only used in case of a MinIO server; connection Endpoint URL for the MinIO storage system |
Bucket | Bucket name in AWS |
Prefix | A folder in a bucket is called prefix in AWS. In the field Prefix only the name of the folder needs to be provided no “/” after the folder name. If a prefix is provided only objects in the folder with the prefix name are listed in the output. |
S3key | Objects matching the given S3key are listed. Unix filename pattern are supported to list only a selection of files:
Example: S3key = test* : matches everything starting with test S3key = test[1-2].txt : matches test1.txt, test2.txt S3key = test[!1].txt: does not match test1.txt S3key = test?.txt: matches test1.txt, test2.txt etc. |
Interval | [10 | 60 | 180] Monitoring Interval in seconds Example: An Interval of 60s means every 60s the bucket is scanned for the key(s) to monitor. |
Example for Amazon S3 Universal Tasks - Monitor Key
Changelog
ut-cs-aws-s3-1.5.25 (2024-05-09)
Enhancements
- Proxy setting for https adjusted to support python 3.11
ut-cs-aws-s3-1.0.0
- Initial Version