|
...
Refer to Changelog for version history information.
Overview
Amazon EMR (Elastic MapReduce) is a cloud-based service offered by Amazon Web Services (AWS) designed to process and analyze large data sets quickly and cost-effectively. Amazon EMR notebooks. AWS EMR Notebooks are Jupyter-based notebooks that are integrated with Amazon EMR, providing execution capabilities and in addition an interactive environment for data exploration and analysis.
This integration provides the ability the start an AWS EMR notebook execution from UAC and optionally monitor it's execution. It also allows Task authors to specify a large variety of notebook execution configuration options.
...
Feature | Description |
---|---|
Start Notebook Execution | Start an AWS EMR Notebook execution with related parameters |
Software Requirements
This integration requires a Universal Agent and a Python runtime to execute the Universal Task.
...
Name | Type | Description | Version Information | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Action | Choice | The action performed upon the task execution. Available options:
Default value is “Start Notebook Execution”. Note that “Notebook” is an entity on AWS that later was renamed to “Workspace”. However in AWS documentation both terms are used. | 1.0.0 | |||||||||
Region | Text | Region for the Amazon Web Service i.e. "us-east-1" When AWS Region is not populated as part of the task definition, during task execution the integration will look for AWS Region on the task execution environment (AWS configuration file or through AWS environment variables). | 1.0.0 | |||||||||
Credentials | Credentials | The Credentials definition should be as follows.
When AWS Credentials is not populated as part of the task definition, during task execution the integration will look for AWS Credentials on the task execution environment. Refer to AWS Credential Configuration Options for more information | 1.0.0 | |||||||||
Assume Role ARN | Large Text | Role ARN of the AWS Assume Role functionality. The Assume Role functionality in AWS (Amazon Web Services) allows a user or service to take on the permissions of another IAM (Identity and Access Management) role temporarily. If field is left empty Role Assumption is notperformed. | 1.0.0 | |||||||||
Service Role | Text | The name or ARN of the IAM role that is used as the service role for Amazon EMR (the Amazon EMR role) for the notebook execution. | 1.0.0 | |||||||||
Execution Engine ID | Text | The unique identifier of the execution engine. For an EMR cluster, this is the cluster ID. | 1.0.0 | |||||||||
Editor ID | Text | The unique identifier of the EMR Notebook to use for notebook execution. | 1.0.0 | |||||||||
Relative Path | Text | The path and file name of the notebook file for this execution, relative to the path specified for the EMR Notebook. For example, if you specify a path of " | 1.0.0 | |||||||||
Notebook Parameters (JSON) | Large Text | Input parameters in JSON format passed to the Amazon EMR Notebook at runtime for execution.
| 1.0.0 | |||||||||
Wait for Completion | Checkbox | A switch that controls whether the UAC task will wait until the Notebook Execution is completed. Default value is unchecked. | 1.0.0 | |||||||||
Polling Interval | Integer | The time (in seconds) between retries for getting the status of Notebook Execution. Mandatory if Wait for Success or Failure is “true” As a best practice, if the Notebook Execution expected completion duration is long, set the polling Interval to a larger value. A short value will trigger frequent checks towards AWS which in the case of long-duration jobs is inefficient in terms of resources. Default value is 15. | 1.0.0 | |||||||||
Max Number of Polls | Integer | Maximum number of polls. Can be used to control the approximate expected duration of the Notebook Execution (in relation also to Polling Interval (sec)) If left empty the UAC Task Instance will poll indefinitely checking whether the Notebook Execution is completed or resulted in failure. If the Maximum Number of Polls is reached the exit code of the Universal Task Instance is 40. Available if Wait for Success or Failure is “true”. | 1.0.0 | |||||||||
Additional Optional Parameters | Checkbox | A switch that controls if any additional optional parameters and configurations should be passed to the notebook execution. Default value is unchecked. | 1.0.0 | |||||||||
Notebook Execution Name | Text | An optional name for the notebook execution. Available if Additional Optional Parameters is “true”. | 1.0.0 | |||||||||
Provide Additional Configuration As | Choice | Specifies how additional configuration options for EMR Notebook execution can be defined on the UAC Task. Available options are:
Available if Additional Optional Parameters is “true”. | 1.0.0 | |||||||||
Additional Configuration (Text) | Large Text | Additional configuration options for the notebook execution.
The default value is a placeholder value meant to serve as a way to inform the user on what the available fields are. It is not meant to be used directly as a default value for an execution of the integration. Task authors can use the default value provided, as a template.
Field is available if Provide Additional Configuration As is “As JSON Text”. | 1.0.0 | |||||||||
Additional Configuration (Script) | Script | Provides the same functionality as Additional Configuration (Text) option with the added benefit of being reusable by saving the configuration as an UAC script. Available if Provide Additional Configuration As is “As JSON UAC Script”. | 1.0.0 |
...
Note |
---|
Backward compatibility is not guaranteed for the content of STDOUT/STDERR and can be changed in future versions without notice |
How To
Import Universal Template
To use the Universal Template, you first must perform the following steps.
This Universal Task requires the Resolvable Credentials feature. Check that the Resolvable Credentials Permitted system property has been set to true.
To import the Universal Template into your Controller, follow these instructions.
When the files have been imported successfully, refresh the Universal Templates list; the Universal Template will appear on the list.
Modifications of this integration, applied by users or customers, before or after import, might affect the supportability of this integration. For more information refer to Integration Modifications.
Configure Universal Task
For a new Universal Task, create a new task, and enter the required input fields.
Integration Modifications
Modifications applied by users or customers, before or after import, might affect the supportability of this integration. The following modifications are discouraged to retain the support level as applied for this integration.
Python code modifications should not be done.
Template Modifications
General Section
"Name", "Extension", "Variable Prefix", "Icon" should not be changed.
Universal Template Details Section
"Template Type", "Agent Type", "Send Extension Variables", "Always Cancel on Force Finish" should not be changed.
Result Processing Defaults Section
Success and Failure Exit codes should not be changed.
Success and Failure Output processing should not be changed.
Fields Restriction Section
The setup of the template does not impose any restrictions, However with respect to "Exit Code Processing Fields" section.Success/Failure exit codes need to be respected.
In principle, as STDERR and STDOUT outputs can change in follow-up releases of this integration, they should not be considered as a reliable source for determining success or failure of a task.
Users and customers are encouraged to report defects, or feature requests at Stonebranch Support Desk.
Document References
This document references the following documents:
Document Link | Description |
---|---|
User documentation for creating, working with, and understanding Universal Templates and Integrations. | |
User documentation for creating Universal Tasks in the Universal Controller user interface. |
Changelog
ue-aws-emr-1.0.0 (2024-06-
0706)
Initial Version