Introduction

From the onset, Universal Extensions have supported the encapsulation of 3rd party modules/packages that are required by the extension for execution. However, prior to UAC 7.4.0, this support was limited to "pure Python" modules that are platform agnostic. Modules containing C libraries and other binary code were not supported.  This was due, in part, to the fact that extensions were always executed directly from the zip archive that contained them - and binary content is not supported in this context.  Furthermore, there was no means to "choose" extension content based on platform or Python runtime environment. 

Starting with UAC 7.4.0.0, Universal Extensions support the ability to embed "Dependency Packages" within the extension archive.  Dependency Packages remove the "pure python" restriction from what can be delivered with a Universal Extension.  This makes it possible to build a wider range of extensions while reducing (or eliminating) any dependencies on the target system where extensions will run - beyond a Universal agent and Python.

What is a Dependency Package

A dependency package is an archive that contains an extension's dependencies.  More specifically, a dependency package contains the dependencies required for a specific Python runtime environment (or range of runtime environments). Dependency package archive files use Python's "weel built package" naming convention:

{extension name}-{extension version}(-{build tag})?-{python tag}-{abitag}-{platform tag}.whl

This allows the Universal agent to select the appropriate dependency package for the Python environment an extension runs in.

How is a Dependency Package Created 

Extension developers use the UIP-CLI utility to generate dependency packages for their extensions.  The process is simple and automated.  Just include the extension's dependencies in the project's requirements.txt file and UIP-CLI will perform the necessary steps with simple build commands. Additional information on using UIP-CLI to generate dependency packages can be found here,

How is a Dependency Package Deployed

Dependency packages are built into an associated Universal extension archive. They are delivered to the target agent along with the extension archive during extension deployment.  When the agent deploys an extension that contains dependency packages, it first selects an appropriate Python interpreter from the list of interpreters defined to that agent.  It then chooses the best (most compatible) dependency package available in the extension archive.  The chosen dependency package is then extracted to the system and setup for extension execution.

When instances of the extension are started by the agent, they will run under the same Python interpreter selected during deployment and the runtime environment will be setup with pathing that includes the dependencies from the deployed dependency package.

Developer Considerations

zip_safe

The zip_safe option in the extension.yml file of an extension must be set to false in order for a Universal agent to extract the extension and deploy a dependency package.

zip_safe: false

By specifying zip_safe: false, the extension is declaring that it is not safe to run the extension in the zipped form (<extension_archive>.zip).

requires_python

For extensions that use dependency packages, the developer should carefully consider the requires_python option specified in the extension's extension.yml file. The extension developer must deliver dependency packages that are compatible with any Python interpreter that could match the specified requires_python statement.

In many cases, a single dependency package may be compatible with many Python versions - possibly all targeted versions.  In such a case, the extension developer would only need to provide one dependency package for each of the target platforms (Windows, linux, etc.).  However, in some cases, the dependencies required by an extension are tightly coupled to a Python version.  The numpy package is a good example of a tightly coupled dependency. Extensions including numpy in the dependency packages would have to deliver a separate dependency package for each version and for each python version - for each platform.  In such a case, the extension developer may choose to be more restrictive in the requires_python statement to limit the number of dependency packages that need to be delivered.  If not, the developer would have to at least understand what the chosen requires_python statement means in terms dependency requirements.

Testing

In order to provide a good experience for the end user of a Universal extension, the developer must ensure that the extensions contain the dependency resources that match the supported Python environments specified in the requires_python statement of the extension.yml file.  As stated in the previous section, the number of dependency packages required for a given extension is entirely dependent on the extension's particular dependencies. The extension developer must determine the compatibility breaking points between the minimum Python version specified in the requires_python clause and the maximum.  At each breaking point, the developer would need to include a dependency package that satisfies the needs of that version.