Page Comparison

B-13831

UIP: Add support for Cancel command within Universal Extension framework

The Universal Integration platform has been enhanced to all cancelling Extension based processes within UIP. The current behavior for cancelling an Extension instance (or any agent based task instance) is as follows:

• Cancel is initiated at the Controller.

• Controller sends JSS-CANCEL message to UAG and transitions the task instance in the Controller to "Cancel Pending".

• UAG receives the JSS-CANCEL message and attempts to terminate process with CSK API function CSProcessCancel().

• If the process has not terminated within 10 seconds, UAG calls CSK API function CSProcessTerminate().

• If the cancel process completes successfully, UAG sends JSS-STATUS(JOB CANCELLED) back to Controller.

• Upon receiving the JSS-STATUS(JOB CANCELLED) message, the Controller transitions the task instance to "Cancelled".

The result is that the Extension instance is terminated without any knowledge of the termination.

In order to give the Extension process a chance to respond to the cancel command, a new extension_cancel() method has been added to the Universal Extension API. This is an optional method that can be implemented by the Extension developer. If implemented, extension_cancel() will be called by the UniversalExtension base class when UAG receives a Cancel command from the Controller.

Within the extension_cancel() method, the Extension developer can perform whatever actions are needed or appropriate before the process is cancelled. However, Extension will not have unlimited time to perform these actions. The specific amount of time available for the cancel operation will be controlled by a UAG configuration option: Cancel Timeout (see below).

With the implementation of this backlog, the cancel behavior for Extension instances changes to the following:

• Cancel is initiated at the Controller.

• Controller sends a JSS-CANCEL message to UAG and transitions the task instance in the Controller to "Cancel Pending".

• UAG receives the JSS-CANCEL message from the Controller and passes an ESS-CANCEL message to the UniversalExtension base class running in the Extension Worker process.

• The UniversalExtension base class calls the extension_cancel() method allowing the Extension to perform cleanup.

• The cancel operation is initiated with the call to extension_cancel() but, does not necessarily end when extension_cancel() completes.

• If the cancel operation does not complete within the available time, the Extension instance will be forcibly cancelled by UAG.

• When the cancel process completes, UAG sends JSS-STATUS(JOB CANCELLED) back to Controller.

• Upon receiving the JSS-STATUS(JOB CANCELLED) message, the Controller transitions the task instance to "Cancelled".

B-13908

UIP: Add support for in-process Dynamic Commands

This backlog adds the ability to run a Dynamic Command within the running process of the task instance on which the command is called.

Currently, Dynamic Commands always run in a separate process.   

Dynamic Commands support custom functionality and it cannot be known what use cases will be implemented.  However, there are three general scenarios that must be considered.

1. Dynamic Command must run in separate process (current functionality).

2. Dynamic Command must run within the process of the associated Extension task instance.

3. It does not matter if the Dynamic Command runs in-process or not.  Run in-process if available.  Otherwise, create new process.

The Dynamic Command functionality was enhanced to support two main execution options that can be selected by the Extension developer:

1. Dynamic Command runs in a separate dedicated process (current functionality): out-of-process.

2. Dynamic Command runs within the process of the associated Extension task instance: in-process.

   1. In-process Dynamic Commands also support the option to be run synchronously or asynchronously.

The Extension developer specifies the execution options at the Universal Template level in the Controller.

B-13821

UIP: Enhance "requires_python" logic to support '*' wildcard in clauses

UIP Extensions now allow usage of Wildcard in clause to implement a simpler way of identifying specific Python requirements.

UIP Extensions allow extension developers to specify if a certain version of Python is required using the requires_python option in the extension.yml file. Currently, the option does not support wildcards which results in unnecessary complexity.

For example, to target all Python 2.7.x binaries, the developer would have to specify the following requires_python: ">=2.7.0,<=2.7.9" . With wildcards, it is simple as requires_python: "==2.7.*

The solution allows wildcards for the == and != operators only.

B-13860

UIP: Refactor logging API

B-13820

UIP: Allow Extension config configuration options to be set at install time or via Broker startupstart-up

Enhance Enhanced UIP to allow certain configuration Options as part of the Install or Broker startup start-up process.

Three configuration options are available to control Universal Extension behavior:

• extension_accept_list

• extension_python_list

• extension_deploy_on_registration

Currently, the only way to set these options is by updating the UAG Server configuration file and then refreshing or restarting the Agent. Because a static configuration is not necessarily desirable for an option like this, providing ways to override the configuration on an ad-hoc or dynamic basis is needed. The ways in which such requirements were satisfied for similar features was to allow them to be set at install time and at Agent startup time, via options in the ubrokerd script and/or the ubroker application’s command line interface..

To satisfy the requirement above, the following enhancements are provided in this release:

• An ability to set the option’s value is provided using the following methods:

  • Setting an environment variable that is inherited by the Universal Broker at startupstart-up.

  • Passing an argument to the Universal Broker script and forwarding that option via the Broker command line.

  • Setting a property during the Universal Agent installation.

B-13822

UIP: Allow usermode installs to discover the Python distribution provided by a system mode-installed Agent

Enhanced UIP to allow a user Mode Agent to take advantage of an, already installed, Python environment provided by a System-install Agent.

During Python discovery (i.e., the process by which UAG Server looks for a Python interpreter that satisfies a Universal Extension’s version requirements), UAG Server always looks for the Python distribution provided by the system-mode Universal Agent install. UAG Server looks for this Python in a file system location that is relative to UAG Server’s installation directory. Because usermode installs do no provide the Python distribution, any Agent-supplied distribution will not be found.

While this problem exists for Linux/Unix usermode installs a change in the UAG Server discovery implementation is only necessary for Windows. If someone wishes to use the Agent’s Python distribution, they can simply take the tarball from the Agent distribution file and extract it under the usermode installation directory. A similar solution is not available for Windows.

Note that the user can also configure the extension_python_list option to look for an Agent-supplied Python distribution in any location. This means that making the change for Windows is simply a change for convenience.

The system mode Universal Agent install for Windows records has its install location in the Windows system registry, via the installation_directory value under HKLM\SOFTWARE\Stonebranch. That is a highly-reliable value that system-mode and usermode installs may use to locate the Agent’s Python distribution.

B-13846

UIP: New Credential "Token" Universal Extension Support

Implemented a new “Credential token” to better support large Stokes provided by certain applications.

Some web/cloud services require authentication tokens that may be 3-4 Kb in length. The requirement is to implement support of token fields in Resolvable Credentials with maximum length at least 4096 bytes. The Credential Token must be passed to Universal Extension and its value must be scrubbed from any visible output of a task instance.

The ‘state’ dictionary of a Universal Extension Task is populated with the new credential variable “[credential variable name].token” along with previously supported ‘*.user’, ‘*.password’, ‘*.pasphrase’ and ‘*.keyLocation’. Total cumulative length of all encrypted fields passed to the task (password + passphrase + token) cannot exceed 65535 characters. The previous limit was 2052 bytes. The limit if used in the command of a regular task is 4096 characters of command length.

B-13819

UIP: Expose configuration options that allow custom Python discovery paths

A new configuration option for UAG Server was implemented for the UA 7.0.0.0 release. This option, extension_python_list, is currently recognized by UAG Server but no information about its use was made available. This was done to reduce the development and testing complexities around exposing any configurable option. However, providing users with the ability to configure this option is needed for usability and flexibility.

Information about this option must be made public and the ability to configure the option in a flexible, secure way must be provided.

To satisfy the requirement above, the following enhancements are provided:

• Information about the option is available in the comments section of the UAG Server configuration file.

• Information about the option is available in the UAG Server template file.

• Information about the option is available in the user documentation.

• An ability to set the option’s value is provided using the following methods:

  • Directly editing the UAG Server configuration file

• Allowing the option to be refreshed via uctl without requiring a restart of the Agent

B-13989

UIP: Provide access to Extension APIs at install time

Simplification of uploading an Extension to the Controller.

So far, with every change, the process of manually building and uploading the Extension has to be repeated.

A tool has been created to make this process faster and convenient.

In addition to building and uploading the Extension, the tool should offer functionality that allows users to:

• Quickly prototype Extensions using starter Extension Templates

• Pull the latest Universal Template source files from the Controller

• Download the full Universal Template package

The Solution is a CLI Utility called uip-cli with a list of parameters associated:

• Quickly prototype Extensions using starter Extension templates:

  • init command is used to initialize the selected Extension template. Additionally, the command has logic to accept variables that are used to configure the selected Extension template.

  • template-list command is used to list all the available starter Extension templates as well as the variables that can be used to configure a given Extension template.

  • Build and upload Extensions and/or Universal Templates:

    • build command is used to build the Extension or the full package (Universal Template + Extension) depending on which one is specified.

    • upload command is used to upload the Extension or the full package to the Controller. The Extension/full package must already be built before calling upload.

    • push command is a combination of build and upload. Each time it is called, the Extension or full package is built AND uploaded.

  • Pull the latest Universal Template source files from the Controller

    • pull command is used to the pull the Universal Template source files from the Controller. The source files include template.json (and template_icon.png, if present). The CLI prints a message indicating what was updated, newly downloaded, or not changed at all.

  • Download the full Universal Template package

    • download command is used to download the full Universal Template package. The CLI prints a message indicating what was updated, newly downloaded, or not changed at all.

B-14255

UIP: Implement init command support for CLI