Disclaimer

Your use of this download is governed by Stonebranch’s Terms of Use, which are available at https://www.stonebranch.com/integration-hub/Terms-and-Privacy/Terms-of-Use/

Overview

Pentaho Data Integration provides powerful ETL (Extract, Transform and Load) capabilities. Universal Controller is integrated to orchestrate the jobs and transformations within Pentaho Data Integration platform via Carte webservice calls.

As an alternative approach, if there is no Carte webserver set-up, it is possible to use the Kitchen & Pan command line utility to execute the jobs & Transformations within the Pentaho Data Integration platform via Windows or Linux Task type in Universal Controller.

The current integration with Pentaho is developed using the Universal Extension functionality in Universal Controller.

The following is the overview of this integration.

UAC communicates with Pentaho Data Integration through the REST-API methods available for the Carte Server.
Universal Controller offers the following functionalities that can be orchestrated within the Pentaho Data Integration Platform:
- Run a Pentaho Data Integration Job where the repository details are already supplied in Carte configuration files.
- Run a Pentaho Data Integration Job from any of the available repository Via Carte web service calls.
- Run a Pentaho Data Integration Job from a *.KJB file.
- Define & Run a Pentaho Integration Job from Universal Controller Script library.
- Execute a Pentaho Data Integration Transformation where the repository details are already supplied in Carte configuration files.
- Run a Pentaho Data Integration Transformation from a *.KTB file.
- Define & Run a Pentaho Integration Transformation from Universal Controller Script library.

Software Requirements

This integration requires a Universal Agent 7.0 and a Python runtime to execute the Universal Task against a Pentaho Data Integration.

Software Requirements for Universal Template and Universal Task

Universal Agent from V7.0 supporting Universal Extension
Requires Python 3.6 or higher. Tested with the Universal Agent bundled Python distribution.
Python modules required:
- requests
- base64
- ElementTree

Software Requirements for Universal Agent

Universal Agent for Linux or Windows Version 7.0.0.0 or later is required.
Universal Agent needs to be installed with python option (--python yes).

Software Requirements for Universal Controller

Universal Controller Version 7.0.0.0 and later.

Software Requirements for the Application to be Scheduled

This Universal Task has been tested with the following Pentaho Data Integration versions:

9.1
Versions from 8.0 can also be supported as it have the same Carte REST-API

Technical Considerations

Universal Agent and Universal Controller from Version 7.0 and later.
Installation of a Universal Agent V7.0 in Pentaho Server.
Pentaho tool must be configured with the Carte Server configuration.
Necessary Python modules installed in the Universal Agent Python library as described in the section above.

Key Features

Feature	Description
Run a Job from Carte configured repository	Trigger a Pentaho Data Integration Job execution ,Where the repository details are already supplied in Carte configuration files.
Run a Job from a Repository	Trigger a Pentaho Data Integration Job execution , Where the Pentaho Repository details can be supplied as a Input in Universal Controller Job definition.
Run Job from file	Run a Pentaho Data Integration Job from a *.KJB file located in the remote host.
Run a Job from UAC Script Library	Define & Run a Pentaho Integration Job from Universal Controller Script library.
Execute a Transformation from a Repository	Trigger a Pentaho Data Integration Transformation execution ,Where the repository details are already supplied in Carte configuration files.
Execute a Transformation from a file	Run a Pentaho Data Integration Job from a *.KTB file located in the remote host.
Run a Transformation from UAC Library	Define & Run a Pentaho Integration Job from Universal Controller Script library.

Import Pentaho Integration Universal Template

To use the Pentaho Universal Extension, you first must perform the following steps:

This Universal Task requires the Resolvable Credentials feature. Check that the Resolvable Credentials Permitted system property has been set to true.
Download the provided ZIP file.
In the Universal Controller UI, select Configuration > Universal Templates to display the current list of Universal Templates.
Click Import Template.
Select the template ZIP file and Import.

When the template has been imported successfully, the Universal Template will appear on the list. Refresh your Navigation Tree to see these tasks in the Automation Center Menu.

Configure Pentaho Integration Universal Task

For the new Universal Task type, create a new task, and enter the task-specific details that were created in the Universal Template.

Field Descriptions for Pentaho Integration Universal Task

Field	Description
Select a Pentaho Function	Select the desired function you would need to perform in Pentaho
Carte server URL	Provide the Carte webserver URL configured in the Pentaho Platform
Carte Credentials	Provide the Carte webserver Credentials for Jobs/Transformation webservice calls
SSL Verify(Carte API Call)	Check if carte web server URL needs to be SSL Verified
Path to SSL Certificate	Provide the path and file name of the SSL Certificate (Needs to be in the utility agent file system)
Is Agent Installed in Carte Server	If you need run a transformation or Job from UAC library then Universal agent needs to be installed in the Pentaho Carte server
Select Pentaho Job or Transformation Script	Select a XML script for Pentaho Job or Transformation from UAC script library
Repository Name	Provide the Pentaho Repository for the Job to be executed
Pentaho Repository Credentials	Provide the Pentaho Repository credentials for the Job Execution REST-API call
Pentaho Job Name	Provide the exact Pentaho job name with path incase of repository and just the job name incase of a file
Job File(Path & Name)	Provide the Pentaho job file name with path and this file should reside in the carte server
Transformation Name	Provide the Pentaho transformation name (*Ensure a unique transformation name when Execute a Transformation from a file)
Transformation File Path & Name	Provide the Pentaho Transformation file name and path
Input Parameters(if amy)	Add if any input parameters to be used for job execution starting with '&' and every parameters to be separated with ampersand symbol(&)
Log Level	Select either INFO or DEBUG OR WARN OR ERROR
Execution ID	Output only field : Displays the Carte execution ID of transformation triggered from Universal Controller
Status	Output only field : Status of the Job/Transformation executing in Carte Web Server
Print Job Log	Check Box if the Job/Transformation execution logs to be printed in Universal Controller STDERR

Examples for Pentaho Integration Universal Tasks

Trigger a Pentaho Data Integration (Carte Server Configured with Repository)

Trigger a Pentaho Data Integration with Repository Details as Input

Launch a Pentaho Job using *.kjb files

Define and Trigger Pentaho Job from Universal Controller Script Library

Pentaho Job Definition Script

Trigger Pentaho Transformation with Input Parameters

Document References

This document references the following documents:

Name	Location	Description
Universal Templates	Universal Templates	User documentation for creating Universal Templates in the Universal Controller user interface.
Universal Tasks	Universal Tasks	User documentation for creating Universal Tasks in the Universal Controller user interface.
Universal Extension	Universal Extension for Universal Controller	User documentation for creating Universal Extension Tasks in the Universal Controller user interface.
Pentaho Rest-API Reference	https://help.hitachivantara.com/Documentation/Pentaho/9.1/Developer_center/REST_API_Reference	Pentaho Developer REST-API Reference