Introduction

This start-up guide describes a use case for how to securely transfer business data located on an on-premise Linux server to an application running on Open Shift, and vice versa, in real-time. As a result, the applications on Open Shift will always have the most up-to-date business data.

The solution also enables the delivery and reception of data from a Windows server, mainframe, or any cloud storage to an application running on Open Shift, and vice versa. For more information, refer to the solution paper Hybrid Cloud File Transfer Solution Paper [8].

Use Case Description

To improve scalability, reduce resource consumption, and shorten the time to develop, test, and roll-out new applications, many IT companies, are currently creating new applications or migrating their applications from their internal core IT environment into an Open Shift environment hosted on-premise and on public clouds.

These new or migrated applications are running in containers in an Open Shift POD. In many cases, they require business data from multiple sources, including on-premise business applications, mainframe, and various public cloud storage systems. Additionally, they both receive and provide data to connected systems, such as SAP business warehouse.

The following use case demonstrates how to securely transfer business data located on a LINUX server to a web application running on Open Shift, in real-time.

USE CASE: “news-flash application”

This sample use case demonstrates how all started instances of a web server for “breaking news” (one POD per web server) are updated in Open Shift with a new webpage. The new webpage HTML files and related pictures are sent from an on-premise server to all running PODs containing a web server instance started for the news-flash application in Open Shift. As a result, all users connected to the new-flash application will see the newly published webpage information.

Key Features

This start-up use case focuses on file transfer from an on-premise server to a group of PODs. The solution can also support many additional scenarios.

The following are its key features:

• Transfer files from any (virtual) Linux/Windows server to an application on Open Shift (and vice versa).
• Transfer files from the mainframe to an application on Open Shift (and vice versa).
• Transfer a file from any cloud storage platform to an application on Open Shift (and vice versa).
• Trigger either a time-based or an event-based file transfer (for example, from a web application using REST APIs).
• End-to-end monitoring of all file transfers (including monitoring of log files).
• Cloud (SaaS) or on-premise solution.
• Enhanced security – validated by regular penetration testing.
• High availability.

Solution Architecture

Universal Automation Center (UAC) is a web-based enterprise scheduler, available as SaaS in the cloud or on-premise.

UAC consists of:

A Universal Agent specifically built for the Open Shift environment is called a Universal Open Shift Agent.

As soon as an agent is installed on a server, it automatically connects to the middleware message bus OMS of Universal Automation Center and is ready to execute remote commands/scripts and file transfers, regardless of whether the agent runs on a server, mainframe, or inside an Open Shift POD.

For applications that provide an API such as SAP, databases, or cloud storage services, no agent is required, as they are scheduled via their corresponding API.

General Description of File Transfer Process

The architecture below outlines how data is transferred from an on-premise server to all instances of an application running on Open Shift. A transfer from the mainframe is performed in a similar way.

Provide business data from an on-premise server to an application running in a POD.

File Transfer Process Steps

How to Get Started

This section outlines how this solution can be deployed in the form of a basic file transfer from a local server to a web application running in Open Shift.

USE CASE: “News-Flash Application”

This sample use case demonstrates how all started instances of a web server for “breaking news” (one POD per webserver) are updated in Open Shift with a new webpage. The new webpage HTML files and related pictures are sent from an on-premise server to all running PODs containing a web server's instance started for the news-flash application in Open Shift. As a result, all users connected to the new-flash application will see the newly published webpage information.

The setup consists of three steps:

1. Configure the file transfer workflow using the Universal Controller Web-GUI.
2. Add the Universal Sidecar container to the application deployment script.
3. Deploy the POD to Open Shift and scale up or down as required.

Prerequisites

To set up this sample use case, the following prerequisites are required:

Configuration Steps

The installation consists of three steps:

1. Configure the file transfer workflow using the Universal Controller web GUI.
2. Add the Universal Sidecar container to the application deployment script.
3. Deploy the POD to Open Shift and scale up or down as required.

Set Up a File Transfer Task in Universal Controller

This scenario requires an update to the homepage of a very simple application (newsflash) consisting of NGNIX web servers. Therefore, the new homepage HTML files and related pictures should be sent from the on-premise Linux server to all running PODs containing a web server instance started for the news-flash application in Open Shift.

Setting up the file transfer task in Universal Controller requires two steps:

1. Define a new Agent Cluster for the Open Shift application “newsflash”.
2. Configure the file transfer task.

Define a New Agent Cluster for the “newsflash” Application

An agent cluster must be configured in Universal Controller for each application in Open Shift. When a POD is started for an application in Open Shift, the Universal Open Shift Agent, which is deployed in a sidecar container to the application POD, will automatically register to the Universal Controller agent cluster related to the application of the started POD.

The agent cluster where all newsflash-related Open Shift agents will register is called AGENT_CLUSTER_APP_NEWSFLASH.

Universal Controller Agent Cluster

Configure the File Transfer Task

A new task must be configured for transferring the files from the on-premise Linux server to all agents assigned to the Universal Controller agent cluster.

Universal Controller File Transfer Task

Universal Controller File Transfer Task script

Description:

• Source Credentials: Linux-OS-Credentials (adjust according to your server credentials)
• Source Linux Server: 168.88.40 (adjust according to your server)
• Source Folder Linux Server: /home/nils/demo/out (adjust according to your server)
• Files to transfer: html, RedHat.png
• Destination Agent Cluster: AGENT_CLUSTER_APP_NEWSFLASH
• Destination Folder in the POD: /podshare/ (this is the mounted POD directory)

The export files of the file transfer task can be found here file transfer task.

Add the Universal Sidecar Container to the Application Deployment Script

Add the Universal Open Shift Agent as a Sidecar Container

To add the Universal Open Shift Agent as a sidecar container to your application, you must add the following section to your application deployment YAML file:

Universal Open Shift Agent deployment YAML

Configure the following three parameters in the deployment file:

*Note: A 30-day demo license key can be obtained from Stonebranch [5].

 Configure a Shared Folder for Application and Sidecar Containers

To make the files received by the Universal Open Shift Agent available to the application in the POD, you must create a shared folder between the application and the sidecar container with the Universal Open Shift Agent.

Open Shift POD shared folder

The following shows the complete deployment script, consisting of the Open Shift web application based on an NGINX web server and the Universal Agent as a sidecar container.

Deployment YAML

The file can be downloaded from GITHUB [6].

Configure Service to Access the Newsflash Application

To make the newsflash application available outside Open Shift, you must create a Service with a NodePort.

Open Shift Service YAML

Open Shift Node port

The Newsflash application will then be accessible via the following IP:

http://192.168.130.11:30763/

Note:

The host IP can be retrieved on a CRC-based Open Shift install via the command: crc ip.

In the described example, it is: 192.168.130.11

Deploy the Open Shift Application

To deploy the application on Open Shift, you only need to deploy the deployment file in the  ‘Deployment Configs’ screen (see below).

Open Shift Deployment Config YAML

In the deployment config file, we defined three replicas. This means that once we press the ‘Create Deployment Config’ button, three instances of our application will be started (= 3 PODs).

These three Universal Open Shift Agents will connect to the Universal Controller agent cluster,  “AGENT_CLUSTER_APP_NEWSFLASH,” which was configured in the deployment script as a parameter and during the set-up in Universal Controller.

In the following screenshot, you can see the three started PODs in Open Shift:

Open Shift

…as well as the related Universal Open Shift Agents, which have registered automatically to the Universal Controller agent cluster:

Universal Controller Agent Cluster with Open Shift Agents

Auto Scaling of POD

If the number of PODs in Open Shift is scaled up, more agents will automatically register for each new POD. Conversely, if the number of PODs is scaled down, the excess agents will automatically be removed from the Universal Controller agent cluster.

Running the File Transfer

When the Universal Open Shift Agents have registered in the Universal Controller agent cluster, they are ready to send and receive files.

The file transfer task in Universal Controller is then launched, initiating the transfer of homepage HTML files, including related pictures from the on-premise Linux server to all started PODs of the newsflash application.

In this example, we triggered the file transfer manually.

Universal Controller File Transfer Task

File Transfer Task Instances

In the screenshot above, you can see that two file transfers were launched. This is because the Universal Controller agent cluster contained two agents, one for each started POD.

As a result, the files are transferred from the Linux server to all assigned PODs and the homepage is updated with the new data:

Newsflash Webpage

The new data can also be viewed in all PODs directly:

Open Shift POD Terminal

Options to Trigger the File Transfer Scenario

In the example above, we triggered the file transfer manually. However, Universal Controller can trigger the file transfers in numerous ways to ensure that the business data that the application requires is always consistent and up to date.

Additional triggers include (but are not limited to):

• File arrival
• Time-based (with support for an internal and external calendar like an SAP calendar)
• Email arrival
• Web services
• Event in a message queue (for example, MQ, JMS)
• SAP Event
• The status of another task/workflow (for example, the start of a new file transfer or if the transfer from last night was successful)

Integration of File Transfer into Microservices Architectures

Any file transfer can be triggered by calling the REST API of Universal Controller.

Essentially, a file transfer workflow can be started from any application, independent of the platform it runs on (for example: virtual server, mainframe, or POD). This single API enables a loosely coupled integration with a microservices architecture.

Security and Auditability

The file transfer protocol outlined above is based on Stonebranch UDM protocol, which encrypts all data and communication channels using TLS1.2 (for example, AES 256 / SHA 384).

Universal Controller’s web GUI real-time reporting functionality provides full auditability through detailed information of all data transfers, including log files.

Document References

Summary and Benefits

The Universal Automation Center, with the introduction of the newly developed Universal Open Shift Agent, enables the secure and reliable transfer of business data located on the mainframe, on cloud storage platforms, or any server, to an application running on Open Shift (and vice versa).

As this example has shown, only three steps are required to configure a secure file transfer to or from any of your Open Shift applications.