Release Notes

Universal Agent release 7.1.0.0 contains the following high-level features. For a complete list of all the included features and fixes please refer to the Universal Agent 7.1.x Maintenance list.

Application Integration

Backlog

Title

Description

B-12809

Update SAP LOW and HIGH data types to support up to 255 characters.

Note

Please make sure you are using the RFC 7.5 Library alongside with the Universal Agent 7.1.0.0 when using this Feature. Any combination other than that might lead to situations where the unavailability of the High option will not be detected and USAP will not identify truncating the field length as SAP enhancements are not present to support the exceeded Length of the Variant Field.

USAP has been modified to support the enhanced variant structures when available. For NW RFC, this is relatively easy. We can check for the existence of the enhanced structures prior to calling the function module and act accordingly.

For classic RFC, it is more difficult. We have no way to determine ahead of time whether the function module in question has been updated with the enhanced variant structures. Therefore, for classic RFC, we will have to assume the enhanced structures are available on an initial call and automatically downgrade to non-enhanced structures on failure.

If the user specifies a variant HIGH/LOW field that exceeds the maximum size supported by the function module involved in an operation, USAP will terminate with exit code 202.

Error on exceeding size limit will be the expected behavior regardless of the variant structure being used (enhanced or original).
The HIGH/LOW size-limit changes based on function module, and whether enhanced structures are supported (see table below for system/function module/size limit expectations.

Universal Integration Platform

Backlog	Title	Description
B-13831	UIP: Add support for Cancel command within Universal Extension framework	The Universal Integration platform has been enhanced to all cancelling Extension based processes within UIP. The current behavior for cancelling an Extension instance (or any agent based task instance) is as follows: Cancel is initiated at the Controller. Controller sends JSS-CANCEL message to UAG and transitions the task instance in the Controller to "Cancel Pending". UAG receives the JSS-CANCEL message and attempts to terminate process with CSK API function CSProcessCancel(). If the process has not terminated within 10 seconds, UAG calls CSK API function CSProcessTerminate(). If the cancel process completes successfully, UAG sends JSS-STATUS(JOB CANCELLED) back to Controller. Upon receiving the JSS-STATUS(JOB CANCELLED) message, the Controller transitions the task instance to "Cancelled". The result is that the Extension instance is terminated without any knowledge of the termination. In order to give the Extension process a chance to respond to the cancel command, a new extension_cancel() method has been added to the Universal Extension API. This is an optional method that can be implemented by the Extension developer. If implemented, extension_cancel() will be called by the UniversalExtension base class when UAG receives a Cancel command from the Controller. Within the extension_cancel() method, the Extension developer can perform whatever actions are needed or appropriate before the process is cancelled. However, Extension will not have unlimited time to perform these actions. The specific amount of time available for the cancel operation will be controlled by a UAG configuration option: Cancel Timeout (see below). With the implementation of this backlog, the cancel behavior for Extension instances changes to the following: Cancel is initiated at the Controller. Controller sends a JSS-CANCEL message to UAG and transitions the task instance in the Controller to "Cancel Pending". UAG receives the JSS-CANCEL message from the Controller and passes an ESS-CANCEL message to the UniversalExtension base class running in the Extension Worker process. The UniversalExtension base class calls the extension_cancel() method allowing the Extension to perform cleanup. The cancel operation is initiated with the call to extension_cancel() but, does not necessarily end when extension_cancel() completes. If the cancel operation does not complete within the available time, the Extension instance will be forcibly cancelled by UAG. When the cancel process completes, UAG sends JSS-STATUS(JOB CANCELLED) back to Controller. Upon receiving the JSS-STATUS(JOB CANCELLED) message, the Controller transitions the task instance to "Cancelled".
B-13908	UIP: Add support for in-process Dynamic Commands	This backlog adds the ability to run a Dynamic Command within the running process of the task instance on which the command is called. Currently, Dynamic Commands always run in a separate process. Dynamic Commands support custom functionality and it cannot be known what use cases will be implemented. However, there are three general scenarios that must be considered. Dynamic Command must run in separate process (current functionality). Dynamic Command must run within the process of the associated Extension task instance. It does not matter if the Dynamic Command runs in-process or not. Run in-process if available. Otherwise, create new process. The Dynamic Command functionality was enhanced to support two main execution options that can be selected by the Extension developer: Dynamic Command runs in a separate dedicated process (current functionality): out-of-process. Dynamic Command runs within the process of the associated Extension task instance: in-process. In-process Dynamic Commands also support the option to be run synchronously or asynchronously. The Extension developer specifies the execution options at the Universal Template level in the Controller.
B-13821	UIP: Enhance "requires_python" logic to support '*' wildcard in clauses	UIP Extensions now allow usage of Wildcard in clause to implement a simpler way of identifying specific Python requirements. UIP Extensions allow extension developers to specify if a certain version of Python is required using the `requires_python` option in the `extension.yml` file. Currently, the option does not support wildcards which results in unnecessary complexity. For example, to target all Python 2.7.x binaries, the developer would have to specify the following `requires_python: ">=2.7.0,<=2.7.`9" . With wildcards, it is simple as `requires_python: "==2.7.*` The solution allows wildcards for the `==` and `!=` operators only.
B-13860	UIP: Refactor logging API	The Universal Extension repository contains a `ExtensionLogger` class that allows users to easily log data. The `UniversalExtension` class in `universal_extension.py` internally creates an instance of `ExtensionLogger` which can be accessed by users using `self.log` in their implementation of the `Extension` class. If the user calls a method, `foo()` from another module called `test.py`, and wants to log some messages in that method, the only way to do so currently is to pass in an instance of `ExtensionLogger`. Instead of passing instances, the user is now able to import the logger in `test.py` and use it in `foo()`. The solution allows users to get access to the logger using the following import: `from universal_extension import logger`. Once imported, the user is able to perform calls such as `logger.info('msg'), logger.error('msg')` etc. Note To preserve compatibility, passing in an instance of the logger still works.
B-13820	UIP: Allow Extension configuration options to be set at install time or via Broker start-up	Enhanced UIP to allow certain configuration Options as part of the Install or Broker start-up process. Three configuration options are available to control Universal Extension behavior: `extension_accept_list` `extension_python_list` `extension_deploy_on_registration` Currently, the only way to set these options is by updating the UAG Server configuration file and then refreshing or restarting the Agent. Because a static configuration is not necessarily desirable for an option like this, providing ways to override the configuration on an ad-hoc or dynamic basis is needed. The ways in which such requirements were satisfied for similar features was to allow them to be set at install time and at Agent startup time, via options in the `ubrokerd` script and/or the `ubroker` application’s command line interface.. To satisfy the requirement above, the following enhancements are provided in this release: An ability to set the option’s value is provided using the following methods: Setting an environment variable that is inherited by the Universal Broker at start-up. Passing an argument to the Universal Broker script and forwarding that option via the Broker command line. Setting a property during the Universal Agent installation.
B-13822	UIP: Allow usermode installs to discover the Python distribution provided by a system mode-installed Agent	Enhanced UIP to allow a user Mode Agent to take advantage of an, already installed, Python environment provided by a System-install Agent. During Python discovery (i.e., the process by which UAG Server looks for a Python interpreter that satisfies a Universal Extension’s version requirements), UAG Server always looks for the Python distribution provided by the system-mode Universal Agent install. UAG Server looks for this Python in a file system location that is relative to UAG Server’s installation directory. Because usermode installs do no provide the Python distribution, any Agent-supplied distribution will not be found. While this problem exists for Linux/Unix usermode installs a change in the UAG Server discovery implementation is only necessary for Windows. If someone wishes to use the Agent’s Python distribution, they can simply take the tarball from the Agent distribution file and extract it under the usermode installation directory. A similar solution is not available for Windows. Note that the user can also configure the `extension_python_list` option to look for an Agent-supplied Python distribution in any location. This means that making the change for Windows is simply a change for convenience. The system mode Universal Agent install for Windows records has its install location in the Windows system registry, via the `installation_directory` value under `HKLM\SOFTWARE\Stonebranch`. That is a highly-reliable value that system-mode and usermode installs may use to locate the Agent’s Python distribution.
B-13846	UIP: New Credential "Token" Universal Extension Support	Implemented a new “Credential token” to better support large tokens provided by certain applications. Some web/cloud services require authentication tokens that may be 3-4 Kb in length. The requirement is to implement support of token fields in Resolvable Credentials with maximum length at least 4096 bytes. The Credential Token must be passed to Universal Extension and its value must be scrubbed from any visible output of a task instance. The ‘fields’ dictionary of a Universal Extension Task is populated with the new credential variable “[credential variable name].token” along with previously supported ‘.user’, ‘.password’, ‘.pasphrase’ and ‘.keyLocation’. Total cumulative length of all encrypted fields passed to the task (password + passphrase + token) cannot exceed 65535 characters. The previous limit was 2052 bytes. The limit if used in the command of a regular task is 4096 characters of command length.
B-13819	UIP: Expose configuration options that allow custom Python discovery paths	A new configuration option for UAG Server was implemented for the UA 7.0.0.0 release. This option, `extension_python_list`, is currently recognized by UAG Server but no information about its use was made available. This was done to reduce the development and testing complexities around exposing any configurable option. However, providing users with the ability to configure this option is needed for usability and flexibility. Information about this option must be made public and the ability to configure the option in a flexible, secure way must be provided. To satisfy the requirement above, the following enhancements are provided: Information about the option is available in the comments section of the UAG Server configuration file. Information about the option is available in the UAG Server template file. Information about the option is available in the user documentation. An ability to set the option’s value is provided using the following methods: Directly editing the UAG Server configuration file Allowing the option to be refreshed via uctl without requiring a restart of the Agent
B-13989	UIP: Provide access to Extension APIs at install time	Simplification of uploading an Extension to the Controller. So far, with every change, the process of manually building and uploading the Extension has to be repeated. A tool has been created to make this process faster and convenient. In addition to building and uploading the Extension, the tool should offer functionality that allows users to: Quickly prototype Extensions using starter Extension Templates Pull the latest Universal Template source files from the Controller Download the full Universal Template package The Solution is a CLI Utility called `uip-cli` with a list of parameters associated: Quickly prototype Extensions using starter Extension templates: `init` command is used to initialize the selected Extension template. Additionally, the command has logic to accept variables that are used to configure the selected Extension template. `template-list` command is used to list all the available starter Extension templates as well as the variables that can be used to configure a given Extension template. Build and upload Extensions and/or Universal Templates: `build` command is used to build the Extension or the full package (Universal Template + Extension) depending on which one is specified. `upload` command is used to upload the Extension or the full package to the Controller. The Extension/full package must already be built before calling upload. `push` command is a combination of `build` and `upload`. Each time it is called, the Extension or full package is built AND uploaded. Pull the latest Universal Template source files from the Controller `pull` command is used to the pull the Universal Template source files from the Controller. The source files include `template.json` (and `template_icon.png`, if present). The CLI prints a message indicating what was updated, newly downloaded, or not changed at all. Download the full Universal Template package `download` command is used to download the full Universal Template package. The CLI prints a message indicating what was updated, newly downloaded, or not changed at all.
B-14255	UIP: Implement init command support for CLI

Architecture

Backlog	Title	Description
B-13768	Move stability checks for file monitor tasks to ops_monitor	Pre-7.1.0.0 implementation of stability checks for Agent File Monitors required requesting updates about current file information from ops_monitor process by uagsrv every second once a matching file was created. The request was implemented as the STAT command sent to the ops_monitor’s STDIN. Handling the response from ops_monitor involved additional internal message exchange between uagsrv process threads. This design resulted in critical performance degradation in case if there are many matching files get created at the same time and uagsrv initiates file stability monitoring for every files. Depending on the number of files there may be long delays in handling the regular tasks execution. In order to improve performance in such conditions, the file stability check functionality was moved to the ops_monitor process entirely, which eliminated most of the IPC message exchange and decreased load on the LSNR thread. That gives more CPU slots for LSNR for handling regular tasks.
B-13885	Suppress ICH408I messages in RACF log	Even though considered as a Warning message only, this enhancement ensures that the warning message is no longer shown on the RACF Log as it only describes the wanted effect that a user does not have the authority he is not supposed to have.
B-13931	OMS Java Client - Add support of service timeout
B-14049	Universal Extension Password Exposed in Agent Cache

Sysplex Failover

Backlog	Title	Description
Please make sure that ALL Sysplex agent components (Primary or Secondary) use the SAME UAG NETNAME definition!
B-13186	Allow a Primary to switch to Secondary	Perform the steps necessary to re-configure a Primary into a Secondary without restarting UAG. This ability can be triggered programmatically and by direct user action through a z/OS console command. If a serious error occurs during the switch-over process, UAG will shut down, as the process' state will be unknown. Command: F <ubroker>,APPL=UAG,SECONDARY
B-13184	Allow a Secondary to switch to Primary	Perform the steps necessary to re-configure a Secondary into a Primary without restarting UAG. This ability can be triggered programmatically and by direct user action through a z/OS console command. If a Primary agent already exists, the command will fail. If a serious error occurs during the switch-over process, UAG will shut down, as the process' state will be unknown. Command: F <ubroker>,APPL=UAG,PRIMARY
B-13282	Automatic fail-over	The automatic fail-over process is triggered when a Primary disconnects from the XCF structure and fail-over is configured. The active Secondary with the highest priority in the fail-over hierarchy becomes the new Primary. The fail-over hierarchy is known by all members. Whenever a member connects to the UAG XCF group, it broadcasts its relative standing in the hierarchy to all members. If two or more members have the same standing in the hierarchy, the member that connected to the structure the earliest is granted the higher standing. Currently, the sysplex_role configuration statement in UBRCFG00 controls the Sysplex mode (none, primary, secondary) for an agent. Since fail-over is a UAG feature, a new configuration statement was added in UAGCFG00. This parameter is used to configure the fail-over behavior of an agent. The automatic_failover statement has two valid values: never - Indicates no automatic fail-over will occur. This is the default. secondary_primary - For Secondary agents only. Indicates this agent is eligible for automatic fail-over.
B-13283	Add a method to configure the fail-over hierarchy	The UAG automatic_failover configuration statement was extended to allow a ranking value to be appended to the secondary_primary value: secondary_primary - This existing value was enhanced to support appending an integer value between 1 and 32. This value indicates the relative ranking of the agent in the fail-over hierarchy. A lower number indicates a higher ranking. If not specified, the ranking is 1. Example: automatic_failover secondary_primary4 # Secondary eligible for fail-over. Ranking is 4.
B-13284	Method to shut down the broker while preventing fail-over	See B-13285 below.
B-13285	Method to shut down the broker while triggering fail-over	The current z/OS STOP command now used to shut down the broker with default (configured) fail-over behavior. The current z/OS MODIFY command was enhanced with new syntax. The new syntax allows shutdown of a Primary broker while specifying if fail-over is required or not and optionally, which Secondary should take over. For Secondary and non-Sysplex systems, the modify shutdown command functions like the STOP command. The FAILOVER and NOFAILOVER keywords will be ignored. e.g.: F <ubroker>,APPL=SHUTDOWN [ [ ,FAILOVER [ ,<sysname> ] ] \| ,NOFAILOVER ] where: <ubroker> is the name of a running broker. SHUTDOWN directs the broker to shut down. FAILOVER, NOFAILOVER specifies whether fail-over is required or not. This will work even if automatic fail-over is not configured. If neither is specified, the command will function like the z/OS STOP command. <sysname> specifies the name of a system in the Sysplex whose broker should become the new Primary. If no UAG in the UAG Sysplex group is running on the specified system, fail-over will not occur. If not specified, the configured fail-over hierarchy will determine which system will become the broker
B-13291	A new Primary can receive tracking data for jobs it hasn't submitted or executed	A new or backup Primary can receive tracking data for jobs it neither submitted, nor are being executed on the system it is running on. Currently, this would result in an error and the job would not be tracked. Only the system which submitted the job and the system executing it has knowledge of the job. For instance, System A submits a job and it executes on System B. System A shuts down and System C takes over as Primary. System C will not recognize the job. System C should consult the UNVJSC dataset for information about jobs which it does not recognize and build the minimum information structures it needs to track the job.
B-13964	Proper licensing / handover when primary or secondary switch

Data Pipeline

Backlog	Title	Description
B-10691	UA: Add explicit FTPS (FTPES) support to UFTP	Enhanced UFTP to support Explicit FTPS(FTPES). The Universal FTP client (UFTP) has been modified to be able to connect to an FTP server that has enabled SSL support but does not necessarily require it as a server configured for implicit SSL support would. A typical use case is one in which a single FTP server, usually – but not necessarily – listening on the default FTP port 21, will support creation of either of the following session types: An SSL/TLS-encrypted session for clients that support or require it An unencrypted, plain-text connection for clients that either have no support for SSL/TLS, or do not require an encrypted session To create a secure, encrypted session, the FTP server presents a public/private key pair in X.509 certificate format to the FTP client (UFTP). The client authenticates this information using a certificate authority (CA). If the certificate information presented by the server cannot be authenticated, the connection fails. The client can set options that changes this behavior. For example, if the client only cares about insecure, plain-text connections, it can choose to bypass certificate (i.e., peer) authentication and accept whatever certificates the server presents. The client can also require host name authentication, where the server’s certificates must contain a host name or IP address that matches the server specified by the client. Note that server options can also be set that disallow plain-text, unencrypted sessions. Note Further details on set-up and security can be found within the documentation.

Development

Backlog

Title

Description

B-13316

Upgrade libSSH2 to 1.9.0 to resolve potential sftp transfer issue

Upgraded the version of libSSH2 from 1.8.2 to 1.9.0 for Universal FTP (UFTP) on those platforms where UFTP was using libSSH2 1.8.2 and/or those platforms on which UFTP provides SSH support.

By definition, this removes HP-UX and z/OS from the targeted platforms.

Browser not supported