Performance Criteria Application and Data Migration

Introduction

This document provides a brief overview or a check-point list for performance tuning in a Micro Focus Server environment. This document will focus on an application and the associated data that is being migrated between two systems (i.e. a ZOS Mainframe System to a Linux, UNIX or Windows System with Micro Focus Enterprise Server).

The abbreviated form of "LUW" will be used in this document when referring to a Linux, UNIX or Windows system as a group of possible systems.

For Batch Processing the performance is measured by elapsed time and is the difference in time from when the job is started and when a job is completed. The wait time for a job that is submitted and scheduled for execution is a scheduling and management task and is not considered as a performance metric. The number of Job Initiators that are configured for a User-Defined Server Instance that does batch processing will affect the wait time. Also, the batch applications (or batch jobs) must be architected to allow for the concurrent running of the batch jobs along with the possible sharing of non-relational Data Files and/or VSAM Data Sets.

For the On Line Transaction Processing (OLTP) the performance is measured in terms of "response time" and the "consistency of the response" time. People that are using a system want a fast response (i.e. sub-second) to their first request and they want the response time for subsequent requests to be consistent (i.e. sub-second response on every request).

The following briefly describes the minimum requirements for Performance Monitoring and Tuning.

If a company is considering their first migration of an application and its associated data they should consider using the professional services of Micro Focus and/or a System Integrator that is familiar with the process.

We have made a significant effort to ensure the documents and software technologies are correct and accurate. We reserve the right to make changes without notice at any time. The function delivered in this version is based upon the enhancement requests from a specific group of users. The intent is to provide changes as the need arises and in a timeframe that is dependent upon the availability of resources.

Copyright © 1987-2025
SimoTime Technologies and Services
All Rights Reserved

Hardware

The foundation for a system starts with the hardware and this will determine the baseline for performance. The following three sub-sections in this document will discuss some of the hardware considerations.

Application Server System

The Application Server is the system where the application programs (batch jobs and/or online transactions) will be executed. The non-relational data would typically be stored on a device that is included within the Application Server System. The non-relational data may include Partitioned Data Sets (PDS's), Generation Data Groups (GDG's), Record Sequential Files (both fixed and variable length records) and VSAM Data Sets (KSDS, ESDS and RRDS).

The Application Sever may be a separate, dedicated physical machine or a virtual server. A dedicated machine with dedicated storage usually provides a higher level of performance with the least amount of tuning effort.

The amount of memory, the number of processors/cores and the processing speed are the critical components of the application server.

A test driver, a monitoring tool and test cases will be basic requirements for a performance tuning project.

Refer to the Micro Focus Enterprise Server section of this document for additional details of configuring an environment that will take advantage of the hardware configuration.

RDBMS Server System

The Relational Data Base Management System (RDBMS) may be deployed on the Application Server or on a separate physical server. Deploying the RDBMS on the same server as the application server does have performance advantages. However, a typical deployment will have the RDBMS on a separate server based on other business and/or technical requirements.

The hardware requirements (this should include the server, storage and connections) for an RDBMS will be based on the recommendations provided by the supplier of the RDBMS and the Data Base Administrator (DBA).

The critical path for performance is typically the connection between the Data Base Server and the Application Server. Also, it is not unusual for a relational data base to be shared with other Servers and Users.

Storage System

The safe and reliable storage of data is the primary business requirement/concern that will drive the selection of storage devices. Next comes the speed at which the data can be retrieved. The following is a list of possible items to consider.

Software

Once the hardware decision has been made the next determination factor for performance is the System and sub-System software.

Operating System Software

When migrating an application between a Mainframe System and a non-Mainframe System the target environment is usually a Linux, UNIX or Windows (LUW) System. It is possible to migrate an application and its associated data between a Mainframe System and an LUW System with little or no changes to the application code (this includes COBOL programs for batch and online processing, JCL Members and BMS Members). This approach will require Micro Focus Enterprise Server.

The configuration options for performance tuning and the monitoring capabilities that are available is dependent on the Operating System that is selected for the RDBMS System and the Application Server.

Note:  If the RDBMS is on a separate server it may use a different operating system than the Application Server. For example, a Linux Operating System may be used on the RDBMS Server and a Windows Operating System may be used on the Application Server.

The leading Operating System providers have basic monitoring capabilities included in the operating system and offer add-on and/or plug-in technologies with more advanced monitoring capabilities.

Relational Data Base (RDBMS)

The RDBMS software may be divided into two categories, the software that manages the physical structures within the Data Base and the utilitarian software that manages the interfaces between the data base and the various users.

The leading RDBMS providers have basic monitoring capabilities included in the products and offer add-on and/or plug-in technologies with more advanced monitoring capabilities.

Micro Focus Enterprise Server

Micro Focus Enterprise Server is available for the Linux, UNIX and Windows Operating Systems. Once the Micro Focus Enterprise Server is installed it will need to be configured to execute in the new environment. The following sections of this document will discuss the tasks involved in defining the new environment and configuring the Application Server or Servers.

Prepare the Environment

    Explore An Enterprise System Model that describes and demonstrates how Applications that were running on a Mainframe System and non-relational data that was located on the Mainframe System were copied and deployed in a Microsoft Windows environment with Micro Focus Enterprise Server.

The following will describe the Environment Variables that may be defined prior to starting a User-Defined Server Instance using a Windows Command File. The "ENV1BASE, Set Common Environment Variables" section of the following link will describe the typical environment variable that may be set for a User-Defined Server Instance. The environment variable that affect performance are noted in the comments.

    Explore How to use Common or Shared Routines that perform repetitive tasks in a consistent manner. This link will provide information about setting common environment variables and many other utilitarian tasks.

Create & Configure a Server

A User-Defined Server Instance typically contains a number of Service Execution Processes (SEP's). SEP's are used to handle (or process) incoming client requests. When the process completes and a response has been returned to a client, the SEP becomes available again to handle another request. If all the SEP's are busy handling requests, incoming requests have to wait until a SEP becomes available.

For processing client requests (or transactions) the number of SEP's available in a User-Defined Server Instance is configurable and will impact performance. The kind of impact depends on a number of variables. You will need to experiment to find out the optimum number of SEP's in any particular user-define server instance.

For processing batch jobs a SEP is defined and allocated for each Job Initiator that is defined. In order for a job to run in an initiator SEP the job class must match the job class supported by the initiator.

The number of SEP's required by an online application is dependent on the workload. On a Mainframe System this would typically be measured in Millions of Instructions per Second (MIP's). In a Windows, UNIX or Linux (LUW) environment MIP's are not used as a measurement of workload. Workload is proportional to the frequency with which user requests arrive, and the required speed of the response to the requests. Also, the number of SEP's to be configured is dependent on the number of cores available, the number of concurrent users and the characteristics of the transactions.

The Shared Memory Pages is the number of shared pages in memory to be reserved for all the processes that run in the Directory Server. Each page is 4 KB long.

The Shared Memory Cushion is the number of 4KB pages that should be reserved from the shared memory pages to allow the server to continue processing when the shared memory allocation is too small for the operational requirements of the server.

Note:  Refer to the Micro Focus documentation for determining the number of Shared Memory Pages and the Shared Memory Cushion

XA Options

The following will provide the details for defining/configuring the XA resources use by the application server. This provides support for compiling and executing application programs that contain EXEC SQL statements.

    Explore how to Define the XA Resources for SQL Access for a user-defined Server Instance that will run under Micro Focus Enterprise Server and allow access to an SQL Data Base.

Transaction Processing Options

The following will provide the details for defining/configuring the resources used by an application server for On-Line Transaction Processing (OLTP). This provides support for compiling and executing application programs that contain EXEC CICS statements.

    Explore how to Configure an Online Region (or Server Instance) that will run under Micro Focus Enterprise Server and execute CICS Transactions.

Batch Processing Options

The following will provide the details for defining/configuring the resources use by an application server for executing batch programs by submitting JCL Members for processing. This provides support for job initiators, JES Spooling and a Catalog for managing non-relational data structures and VSAM Data Sets.

    Explore how to Configure a Batch JES sub-system (or Server Instance) that will run under Micro Focus Enterprise Server and execute batch jobs using JCL.

Third Party Technologies

Micro Focus and various third-party vendors provide a variety of integration and support options. The first thing to validate is the availability of the third-party technology on the Operating System of preference.

Application

The characteristics of the application (Batch Processing, Transaction Processing or both) are the significant factors that will need to be understood when doing performance tuning. If the daytime hours are dedicated to running a Transaction Server (Online) and the late-night, early-morning hours are dedicated to a batch processing application then each may be tuned (or configured) to leverage the system resources to their advantage.

However, in recent years a batch processing window of time has been shrinking or has been eliminated. The Transaction Server needs to be available 24x7 in order to meet business requirements.

In this new environment Relational Data Base Management Systems (RDBMS's) have improved the capabilities of sharing data between batch processing cycles and transaction processing environments. The batch processing may need to be scheduled when the Transaction (or online) Server is in a minimal use period of time.

If non-relational data files and/or VSAM Data Sets are used by the application then certain files and/or VSAM Data Sets may be taken offline for a few minutes of time while other files or data sets may be shared with a batch processing application. during this period. Some online functions may not be available but critical business function will stay online with a possible reduction in performance. This window of time and possible reduction in performance will need to be kept to a minimum. The batch processing will need to be scheduled when the Transaction (or online) Server is in a minimal use period of time.

Compile Options and Directives

The Micro Focus compiler has the capability of producing a variety of executable members. A general guideline is to use the compiler option that generates native code for the target environment. For Windows the would be a .GNT or .DLL member. For Linux and UNIX this would be a .GNT or .SO member.

The following link describes four groups of compiler directive settings that include Batch programs (with and without embedded SQL statements) and Transaction programs (With and without embedded SQL statements)

    Explore the Compiler Directives available for the Micro Focus COBOL technologies.

Note:  Refer to the Micro Focus documentation for additional details about compiler directives.

Execution Time

    Explore An Enterprise System Model that describes and demonstrates how Applications that were running on a Mainframe System and non-relational data that was located on the Mainframe System were copied and deployed in a Microsoft Windows environment with Micro Focus Enterprise Server.

Data, non-Relational

The non-relational data may include Partitioned Data Sets (PDS's), Generation Data Groups (GDG's), Record Sequential Files (both fixed and variable length records) and VSAM Data Sets (KSDS, ESDS and RRDS).

Size, Format, Access and Location

The file size and file format are determined by the application design and business requirements. From a performance tuning perspective this cannot be changed and it will be necessary to design and configure a system that will meet the business requirements. The speed of the hardware (especially the Storage Device or Devices) and the location of the files are critical factors that can be adjusted.

File Configuration Options (EXTFH.CFG)

The following shows the entries for the EXTFH Configuration File.

[XFH-DEFAULT]
filemaxsize=8
idxformat=8
filepointersize=8
INDEXCOUNT=32
IGNORELOCK=ON
READSEMA=OFF
USEVSAMKEYDEFS=OFF

Note:  The filemaxsize, idxformat and filepointersize enable the function that will support large files exceeding two (2) gigabytes.

The default filename for the File Handler configuration file is extfh.cfg but you can use a different filename by setting the EXTFH environment variable, for example:

set EXTFH=c:/mydir/test.cfg

The preceding statement sets the filename to c:/mydir/test.cfg.

    Explore How to Configure a Micro Focus Server to support Large Files with the physical file sizes exceeding four (4) gigabytes.

File Location and ES_ALLOC_OVERRIDE

When a Region (or Server) is configured within Micro Focus Enterprise Server to run batch jobs (i.e. submit JCL) a default directory is defined and used for placement of new physical files that are allocated by a job. This is an example of how to catalog a new data set and override the base configuration and place the physical file in an alternate directory that may be on a different physical device. The technique provides the capability to key off any portion of a catalog record to control the target directory where the physical files will reside. By placing input and output files on different physical devices the I/O contention and/or I/O wait times may be reduced resulting in better performance.

The location of the configuration file must be defined to Enterprise Server by setting the following environment variable.

c:\> set ES_ALLOC_OVERRIDE=d:\dirname\CATMAPA1.cfg 

    Explore how to catalog a new data set and override the base configuration and place the physical file in an alternate directory.

SORT

The Micro Focus SORT program provides a high-level of support for applications being migrated from a Mainframe System to an LUW (Linux, UNIX or Windows) System and it works quite well as delivered for small files. As the files increase in size a user may want to consider allocating more resources to the sorting process to improve performance. The following shows a couple of environment variables that may be used to increase the performance of sorting larger files.

rem  * For Sort Resource Allocation and Performance
     set SORTSCHEME=1
     set SORTSPACE=500000000
     set TMP=D:\SORTWORK

Also, the physical location of the file to be sorted (SORTIN) and the new sorted file (SORTOUT) and the physical location of the sort work files will affect the performance or run time of a sort process.

SPOOL and SYSOUT

The SPOOL and SYSOUT files are written to the "Default Allocated Dataset Location" that is defined when configuring the JES parameters for a server (or JES Region). Overriding this default location has functional and operational advantages by simply separating the location of these files from the application data files. If the location of the directory for the SPOOL and SYSOUT files is located on a separate physical drive from the system functions and user data this could reduce contention for access to the physical drive and improve performance.

Sharing and Concurrency

The sharing and concurrent use of system resources should be treated as a requirement with certain requests having priority over other requests. For example, On Line Transactions will be given a higher priority than batch jobs. Batch jobs may be scheduled to execute at times when the on line requests are at their lowest.

Multiple Users

The number of users and the types of users needs to be defined. This is usually determined by the requirements of the business and sets the objectives for System performance. All users are not alike, some will make casual use of a system while others (referred to as Power Users) will place heavy demands on the system. Also, the number of users and the subsequent level of system demand will vary on a daily basis.

Allocating the proper number of cores and providing the proper number of SEPs that will handle the peak system demands will take some analysis and testing experience.

Batch Processing

Batch applications typically do consecutive processing of large amounts of data in a single job containing multiple job steps. A batch job may scan a master file, extract data to a sequential file, sort the file and may produce large amounts of detailed output and/or smaller amounts of summary information.

If batch processing will be executed in a dedicated batch processing window (i.e. the On Line Transaction Processing server is shut down) then system resources can be dedicated and configured to complement a batch environment vs. a transaction environment. This approach should deliver the shortest run times for the batch jobs. When the batch jobs complete the batch system (or Server) may be shutdown and all system resources made available to a Transaction Server.

However, if the Batch Jobs are going to be processed concurrently with the On Line Transaction Processing then system resources and data will need to be shared between the processing cycles. With this type of environment the Transaction Processing is typically the primary focus of performance tuning efforts. The system resources allocated to the Batch Job processing are secondary and may be limited in order to minimize the impact on the Transaction Processing.

Manually reviewing the JES output spool files to validate successful job execution and calculating elapsed time from the job start and job ending times may be a viable approach if the testing and review process has a few small jobs. However, this can become very time-consuming and boring for a large number of jobs with multiple job steps. The following link describes a utility program that will read the Micro Focus "JES" logs and summarize the output with calculated elapsed times.

    Explore How to Access the JES Output produced by the Micro Focus Server and calculate elapsed job time and post information to a user specified file.

On Line Transaction Processing

The rate at which information is delivered to users is measured in the number of Transactions per Second (TPS). The items that affect the TPS rate are the types of transactions, number of concurrent user requests, the quantity of data retrieval and the data access methods.

Long running transactions may require extra attention as to how and when they are processed. The User-Defined Server Instance will need to be configured with an adequate number of SEP's to allow for the concurrent processing of the long-running transactions and maintain availability of a number of SEP's for the terminal-based and/or short running transactions.

If all the SEP's are consumed by long running transactions then new transaction requests will need to wait until a SEP becomes available.

Summary

This document provides a brief overview or a check-point list for performance tuning in a Micro Focus Server environment. This document may be used as a tutorial for new programmers or as a quick reference for experienced programmers.

In the world of programming there are many ways to solve a problem. This documentation and software were developed and tested on systems that are configured for a SIMOTIME environment based on the hardware, operating systems, user requirements and security requirements. Therefore, adjustments may be needed to execute the jobs and programs when transferred to a system of a different architecture or configuration.

SIMOTIME Services has experience in moving or sharing data or application processing across a variety of systems. For additional information about SIMOTIME Services or Technologies please contact us using the information in the  Contact or Feedback  section of this document.

Software Agreement and Disclaimer

Permission to use, copy, modify and distribute this software, documentation or training material for any purpose requires a fee to be paid to SimoTime Technologies. Once the fee is received by SimoTime the latest version of the software, documentation or training material will be delivered and a license will be granted for use within an enterprise, provided the SimoTime copyright notice appear on all copies of the software. The SimoTime name or Logo may not be used in any advertising or publicity pertaining to the use of the software without the written permission of SimoTime Technologies.

SimoTime Technologies makes no warranty or representations about the suitability of the software, documentation or learning material for any purpose. It is provided "AS IS" without any expressed or implied warranty, including the implied warranties of merchantability, fitness for a particular purpose and non-infringement. SimoTime Technologies shall not be liable for any direct, indirect, special or consequential damages resulting from the loss of use, data or projects, whether in an action of contract or tort, arising out of or in connection with the use or performance of this software, documentation or training material.

Downloads and Links

This section includes links to documents with additional information that are beyond the scope and purpose of this document. The first group of documents may be available from a local system or via an internet connection, the second group of documents will require an internet connection.

Note: A SimoTime License is required for the items to be made available on a local system or server.

Current Server or Internet Access

The following links may be to the current server or to the Internet.

Note: The latest versions of the SimoTime Documents and Program Suites are available on the Internet and may be accessed using the icon. If a user has a SimoTime Enterprise License the Documents and Program Suites may be available on a local server and accessed using the icon.

    Explore How to Calculate Elapsed Processing Times that will be used as a baseline reference when processing small, medium and large data files. This test case is intended for the Micro Focus Server environment and requires the SIMOTIME Library to be installed.

    Explore how to calculate elapsed times for batch jobs using the JES Log files produced by Micro Focus Enterprise Server.

    Explore The ASCII and EBCDIC Translation Tables. These tables are provided for individuals that need to better understand the bit structures and differences of the encoding formats.

    Explore The File Status Return Codes that are used to interpret the results of accessing VSAM data sets and/or QSAM files.

Internet Access Required

The following links will require an internet connection.

A good place to start is The SimoTime Home Page for access to white papers, program examples and product information. This link requires an Internet Connection

Explore The Micro Focus Web Site for more information about products (including Micro Focus COBOL) and services available from Micro Focus. This link requires an Internet Connection.

Glossary of Terms

    Explore the Glossary of Terms for a list of terms and definitions used in this suite of documents and white papers.

Comments or Feedback

This document was created and is maintained by SimoTime Technologies. If you have any questions, suggestions, comments or feedback please use the following contact information.

We appreciate hearing from you.

Company Overview

SimoTime Technologies was founded in 1987 and is a privately owned company. We specialize in the creation and deployment of business applications using new or existing technologies and services. We have a team of individuals that understand the broad range of technologies being used in today's environments. Our customers include small businesses using Internet technologies to corporations using very large mainframe systems.

Quite often, to reach larger markets or provide a higher level of service to existing customers it requires the newer Internet technologies to work in a complementary manner with existing corporate mainframe systems. We specialize in preparing applications and the associated data that are currently residing on a single platform to be distributed across a variety of platforms.

Preparing the application programs will require the transfer of source members that will be compiled and deployed on the target platform. The data will need to be transferred between the systems and may need to be converted and validated at various stages within the process. SimoTime has the technology, services and experience to assist in the application and data management tasks involved with doing business in a multi-system environment.

Whether you want to use the Internet to expand into new market segments or as a delivery vehicle for existing business functions simply give us a call or check the web site at http://www.simotime.com