Disclaimer

Your use of this download is governed by Stonebranch's Terms of Use.

Version Information

Refer to Changelog for version history information.

Overview

Prefect is an open-source workflow management system designed to orchestrate data pipelines in Python. This integration provides the capability to trigger a Prefect Flow Run from UAC and monitor its execution, providing useful metadata.

Key Features

Software Requirements

This integration requires a Universal Agent and a Python runtime to execute the Universal Task.

Supported Actions

Action: Prefect Cloud - Launch Flow Run

This action allows you to launch a Flow Run on Prefect Cloud. The task author has to provide the required Credentials, the Workspace ID and the Deployment ID that relates to the Flow, and some additional parameters related to the flow execution (more information can be found in the Fields Description section). The task author can decide whether the task will just launch the flow, or in addition, wait until a final success or failure state is reached. The last execution Flow Run metadata can be published as part of the Extension Output, while the Flow Run ID, the Flow Run Name and the Flow Run State can be visible directly as output fields during task instance execution.

Action Output

{
    "exit_code": 0,
    "status_description": "Task executed successfully.",
    "invocation": {
        "fields": {...}
    },
    "result": {
        "flow_run_info": {
            "id": "c5feefe0-e932-4f1e-9055-03155561938d",
            "created": "2024-04-30T09:41:40.633550+00:00",
            "updated": "2024-04-30T09:41:40.628595+00:00",
            "name": "mydummyname",
            "flow_id": "6342cf22-2baa-4b4f-8c7f-85d731055af8",
            "state_id": "70736b0e-75e2-4156-8bca-8a21f4c538e4",
            "deployment_id": "ef8baf82-a14b-472f-9f8d-5424b66f232d",
            "work_queue_id": null,
            "work_queue_name": null,
            "flow_version": null,
            "deployment_version": "49e24154c00009d471e26e915271e743",
            "parameters": {
                "fail": false,
                "sleeptime": 5
            },
            "idempotency_key": null,
            "context": {
                "context1": "value1",
                "context2": "value2"
            },
            "empirical_policy": {
                "max_retries": 0,
                "retry_delay_seconds": 0,
                "retries": null,
                "retry_delay": null,
                "pause_keys": [],
                "resuming": false
            },
            "tags": [
                "telis 1",
                "telis 2",
                "testing"
            ],
            "parent_task_run_id": null,
            "state_type": "SCHEDULED",
            "state_name": "Scheduled",
            "run_count": 0,
            "expected_start_time": "2024-04-30T09:41:40.633473+00:00",
            "next_scheduled_start_time": "2024-04-30T09:41:40.633473+00:00",
            "start_time": null,
            "end_time": null,
            "total_run_time": 0,
            "estimated_run_time": 0,
            "estimated_start_time_delta": 0.049332,
            "auto_scheduled": false,
            "infrastructure_document_id": null,
            "infrastructure_pid": null,
            "created_by": {
                "id": "34e9ed52-f547-408e-b53d-02968ba97d64",
                "type": "USER",
                "display_value": "theta457-edison"
            },
            "work_pool_id": null,
            "work_pool_name": null,
            "state": {
                "id": "70736b0e-75e2-4156-8bca-8a21f4c538e4",
                "type": "SCHEDULED",
                "name": "Scheduled",
                "timestamp": "2024-04-30T09:41:40.633516+00:00",
                "message": null,
                "data": null,
                "state_details": {
                    "flow_run_id": "c5feefe0-e932-4f1e-9055-03155561938d",
                    "task_run_id": null,
                    "child_flow_run_id": null,
                    "scheduled_time": "2024-04-30T09:41:40.633473+00:00",
                    "cache_key": null,
                    "cache_expiration": null,
                    "untrackable_result": false,
                    "pause_timeout": null,
                    "pause_reschedule": false,
                    "pause_key": null,
                    "run_input_keyset": null,
                    "refresh_cache": null,
                    "retriable": null,
                    "transition_id": null,
                    "task_parameters_id": null
                }
            },
            "job_variables": {
                "var1": "value1",
                "var2": "value2"
            }
        },
        "errors": []
    }
}

{
    "exit_code": 0,
    "status_description": "Execution Failed: Flow run Reached CRASHED State",
    "invocation": {
        "fields": {...}
    },
    "result": {
        "flow_run_info": {
            "id": "4cbbeb73-2438-4155-8e20-196e5f53c558",
            "created": "2024-04-30T09:47:08.664604+00:00",
            "updated": "2024-04-30T09:47:22.588768+00:00",
            "name": "mydummyname",
            "flow_id": "6342cf22-2baa-4b4f-8c7f-85d731055af8",
            "state_id": "969d5d10-9e43-48f3-ba36-b81c9808687f",
            "deployment_id": "ef8baf82-a14b-472f-9f8d-5424b66f232d",
            "work_queue_id": null,
            "work_queue_name": null,
            "flow_version": "49e24154c00009d471e26e915271e743",
            "deployment_version": "49e24154c00009d471e26e915271e743",
            "parameters": {
                "fail": true,
                "sleeptime": 5
            },
            "idempotency_key": null,
            "context": {
                "context1": "value1",
                "context2": "value2"
            },
            "empirical_policy": {
                "max_retries": 0,
                "retry_delay_seconds": 0,
                "retries": 0,
                "retry_delay": 0,
                "pause_keys": [],
                "resuming": false
            },
            "tags": [
                "testing",
                "telis 1",
                "telis 2"
            ],
            "parent_task_run_id": null,
            "state_type": "CRASHED",
            "state_name": "Crashed",
            "run_count": 1,
            "expected_start_time": "2024-04-30T09:47:08.664475+00:00",
            "next_scheduled_start_time": null,
            "start_time": "2024-04-30T09:47:17.117950+00:00",
            "end_time": "2024-04-30T09:47:22.582148+00:00",
            "total_run_time": 5.464198,
            "estimated_run_time": 5.464198,
            "estimated_start_time_delta": 8.453475,
            "auto_scheduled": false,
            "infrastructure_document_id": null,
            "infrastructure_pid": null,
            "created_by": {
                "id": "34e9ed52-f547-408e-b53d-02968ba97d64",
                "type": "USER",
                "display_value": "theta457-edison"
            },
            "work_pool_id": null,
            "work_pool_name": null,
            "state": {
                "id": "969d5d10-9e43-48f3-ba36-b81c9808687f",
                "type": "CRASHED",
                "name": "Crashed",
                "timestamp": "2024-04-30T09:47:22.582148+00:00",
                "message": "Execution was aborted by Python system exit call.",
                "data": null,
                "state_details": {
                    "flow_run_id": "4cbbeb73-2438-4155-8e20-196e5f53c558",
                    "task_run_id": null,
                    "child_flow_run_id": null,
                    "scheduled_time": null,
                    "cache_key": null,
                    "cache_expiration": null,
                    "untrackable_result": false,
                    "pause_timeout": null,
                    "pause_reschedule": false,
                    "pause_key": null,
                    "run_input_keyset": null,
                    "refresh_cache": null,
                    "retriable": null,
                    "transition_id": "bcd4ea0a-f7e5-40ef-a492-5ce17bdd8651",
                    "task_parameters_id": null
                }
            },
            "job_variables": {
                "var1": "value1",
                "var2": "value2"
            }
        },
        "errors": [
            "Flow run Reached CRASHED State"
        ]
    }
}

Configuration Examples

Example: Simple Launch Flow Execution using Names

Simple Launch Flow Execution using Workspace ID or Name and Deployment ID or Name as dynamic choice fields to retrieve the necessary names from the Prefect Cloud Platform.

Example: Simple Launch Flow Execution using IDs

Simple Launch Flow Execution using Workspace ID or Name and Deployment ID or Name as IDs that can be copy-pasted from Prefect Console. For example purposes, dummy IDs are used.

Example: Launch Flow by providing Flow Parameters and Tags

In this case, it is assumed that the Flow expects a Boolean and an integer parameter.

Example: Launch Flow and wait until the Flow ends in Success or in Failure

The latest Flow Run metadata is configured to be included as part of the Extension Output.

Importable Configuration Examples

This integration provides importable configuration examples along with their dependencies, grouped as Use Cases to better describe end to end capabilities. 

Initial Preparation Steps

• STEP 1: Go to Stonebranch Integration Hub and download "Prefect Cloud" and  "UAC Utility: Email" and integrations. Extract the downloaded archives on separate directories in a local drive.
• STEP 2: Locate and import the above integrations to the target Universal Controller. For more information refer to the How To section in this document.
• STEP 3: For "Prefect Cloud", inside the directory named "configuration_examples" you will find a list of definition zip files. Upload them one by one respecting the order presented below, by using the "Upload" functionality of Universal Controller: 
  • variables.zip
  • credentials.zip
  • scripts.zip
  • tasks.zip
  • workflows.zip

• STEP 4: Update the uploaded UAC Credential entities introduced with proper Account ID/Account Key for "Prefect Cloud" and Client ID / Client Secret for "UAC Utility: Email".
• STEP 5: Update the UAC global variables introduced with variables.zip file. Their name is prefixed with "ue_prefect". Review the descriptions of the variables as they include information on how should be populated.

How to "Upload" Definition Files to a Universal Controller

The "Upload" functionality of Universal Controller allows Users to import definitions exported with the "Download" functionality.

Login to Universal Controller and:

• STEP 1: Click on "Tasks" → "All Tasks"
• STEP 2: Right click on the top of the column named "Name"
• STEP 3: Click "Upload..."

In the pop-up "Upload..." dialogue:

• STEP 1: Click "Choose File".
• STEP 2: Select the appropriate zip definition file and click "Upload".
• STEP 3: Observe the Console for possible errors.

Use Case 1: Launch Flow Run, Wait for Success or Failure, send Email

Description

This workflow is designed to launch Prefect Flow Run, wait for Success or Failure and based on the state of execution, and send an email with the appropriate content. 

The tasks configured demonstrate the following capabilities among others:

• Integration with Prefect Cloud: Launch and execution of Prefect Flow runs within a workflow.
• Integration with another extension in a workflow: Use another extension, such as "UAC Utility: Email" to showcase interoperability between extensions.
• Result Processing: Demonstrating the results of execution of a flow run in more user-friendly way.

The components of the solution are described below:

1. "UC1: Launch Flow Run and Wait for Success or Failure" - This is the starting task of the workflow. It launches a Prefect Flow Run in the designated workspace and deployment and waits for the execution to finish.
2. "UC1: Send Email - Successful Flow Run" - When the "UC1: Launch Flow Run and Wait for Success or Failure" task executes successfully, an email with appropriate content is sent to the user.
3. "UC1: Send Email - Failed Flow Run" - When the "UC1: Launch Flow Run and Wait for Success or Failure" task fails, an email with appropriate content is sent to the user.

Example of email content that is received when the Prefect task fails.

How to Run

Execution Steps:

• STEP 1: Launch Workflow "UC1: Launch Flow Run and Wait for Success or Failure".
• STEP 2: After the workflow finishes, depending on the state of the executed flow run, an email is received which can be viewed by the user.

Expected Results:

• Prefect Flow Run is launched and executes in success or failure.
• Email message is received based on the state of the execution of the flow run.

Input Fields

Output Fields

Cancelation and Rerun

In case of cancellation, the Flow Run is not canceled on Prefect. In case of Re-Run, the Flow Run will be re-executed from scratch.

Exit Codes 

STDOUT and STDERR

STDOUT is empty, while STDERR provides additional information to the user.

How To

Import Universal Template

1. This Universal Task requires the Resolvable Credentials feature. Check that the Resolvable Credentials Permitted system property has been set to true.

2. Import the Universal Template into your Controller:

   1. Extract the zip file, you downloaded from the Integration Hub.

   2. In the Controller UI, select Services > Import Integration Template option.

   3. Browse to the “export” folder under the extracted files for the ZIP file (Name of the file will be unv_tmplt_*.zip) and click Import.

   4. When the file is imported successfully, refresh the Universal Templates list; the Universal Template will appear on the list.

Modifications of this integration, applied by users or customers, before or after import, might affect the supportability of this integration. For more information refer to Integration Modifications.

Configure Universal Task

For a new Universal Task, create a new task, and enter the required input fields.

Integration Modifications 

Modifications applied by users or customers, before or after import, might affect the supportability of this integration. The following modifications are discouraged to retain the support level as applied for this integration.

• Python code modifications should not be done.

• Template Modifications

  • General Section

    • "Name", "Extension", "Variable Prefix", and "Icon" should not be changed.

  • Universal Template Details Section

    • "Template Type", "Agent Type", "Send Extension Variables", and "Always Cancel on Force Finish" should not be changed.

  • Result Processing Defaults Section

    • Success and Failure Exit codes should not be changed.

    • Success and Failure Output processing should not be changed.

  • Fields Restriction Section 

                     The setup of the template does not impose any restrictions. However, concerning the "Exit Code Processing Fields" section.

                      i. Success/Failure exit codes need to be respected.

                      ii. In principle, as STDERR and STDOUT outputs can change in follow-up releases of this integration, they should not be considered as a reliable source for determining the success or failure of a task.

Users and customers are encouraged to report defects, or feature requests at Stonebranch Support Desk.

Document References

This document references the following documents:

Changelog

ue-prefect-1.0.2 (2025-08-28)

• Fixes: Fix incomplete resource fetching by adding pagination for workspaces and deployments. (#48558, #138254)

ue-prefect-1.0.1 (2025-01-23)

• Enhancements: Enhanced log messages for API calls and added SSL configuration analysis messages. (#43417, #126846)

ue-prefect-1.0.0 (2024-07-19)

Initial Release supporting Flow Run executions on Prefect Cloud.