Data Structures & Files
Information Gathering for a File Repository

Table of Contents

v-24.01.01 - datrak01.htm

	Introduction
	Gather non-Relational Data Info
	Application Analysis Tools
	Scan JCL, PROC's & Control Files
	LISTCAT & DUMPCAT Utility Programs
	Spread Sheet Information
	Micro Focus Catalog Extract
	Ongoing Growth of FPMASTER.DAT
	Online Access of FPMASTER.DAT
	Build a Catalog for Micro Focus
	Create Catalog Entries
	Catalog Entries for PDS's
	Catalog Entries for GDG's
	Catalog Entries for Sequential Files
	Catalog Entries for VSAM Data Sets
	Repository Record Structure
	Summary
	Software Agreement and Disclaimer
	Downloads and Links
	Current Server or Internet Access
	Internet Access Required
	Glossary of Terms
	Contact or Feedback
	Company Overview

The SimoTime Home Page

Introduction

Creating and maintaining an inventory of data files can be a challenge. The challenge is not with the degree of difficulty or complexity. The challenge is with the realization of the value, a commitment to the task and elevating the priority of doing a very boring task. However, once it is accomplished it becomes a very worthwhile repository of information that can be used in the planning, development, testing and production environments.

Information about the characteristic (or properties) of a Data File, Data Structure (such as a PDS or GDG) or VSAM Data Set is not always readily available. In today's world when application processing is being moved between systems and data is being transferred, shared, converted and compared an inventory of data files can be quite helpful.

This example will use a Windows System running Micro Focus Enterprise Studio to create a Key-Sequenced-Data-Set (or VSAM, KSDS) containing an inventory of Partitioned Data Sets, Generation Data Groups, Sequential Files and VSAM Data Sets. Once the inventory of data sets has been created it will be possible to use this information to build a catalog for Micro Focus Mainframe Express or Micro Focus Studio/Server environments.

Note: This document is currently a Work in Progress

We have made a significant effort to ensure the documents and software technologies are correct and accurate. We reserve the right to make changes without notice at any time. The function delivered in this version is based upon the enhancement requests from a specific group of users. The intent is to provide changes as the need arises and in a timeframe that is dependent upon the availability of resources.

Gather non-Relational Data Info

There are numerous ways to gather information about the existence of files and the file properties. This section will discuss various methods for collecting information about files used by an application.

1	Create a Spreadsheet for the Inventory of the Partitioned Data Sets (PDS's)
2	Create a Spreadsheet for the Inventory of the Generation Data Groups (GDG's)
3	Create a Spreadsheet for the Inventory of the Sequential Files
4	Create a Spreadsheet for the Inventory of the VSAM Data Sets

Build Four Excel Spreadsheets for the Data Set Inventory and Associated Properties

Application Analysis Tools

WIP2

Scan JCL, PROC's & Control Files

WIP2

LISTCAT & DUMPCAT Utility Programs

WIP2

@echo OFF
rem  *
rem  * ESA, Enterprise Server Administration
rem  *      Catalog List Utility
rem  *
rem  * CATGET01 identifies the CATALOG.DAT file that will be used as input
rem  * CATTXT01 identifies the ASCII/Text file to be used as output
rem  * CATCTL01 identifies the control file, if null then default to list
rem  *
     call ..\..\ENV1BASE
     set CATGET01=%BASECAT%\CATALOG.DAT
     set CATTXT01=%BaseSYS%\LOGS\SYSOUT_SYS1CATLIST.TXT
rem  *
rem  * By setting the CATCTL01 to a null value the catalog list program will
rem  * use the program defaults and produce a list file with the binary
rem  * values removed prior to writing to the text file.
     set CATCTL01=%BaseSYS%\PARMLIB\CATLIST1.CTL
rem  *
     echo *******************************************************SYS1CATLIST
     echo Starting JobName SYS1CATLIST, User is %USERNAME%
rem  *
     run CBLCATC2
     if not "%ERRORLEVEL%" == "0" goto EOJNOK
     echo The Catalog list will be displayed in a separate window...
     START NotePad %CATTXT01%
rem  *
:EOJAOK
     echo Finished JobName SYS1CATLIST, User is %USERNAME%
     goto END
rem  *
:EOJNOK
     echo ABENDING JobName SYS1CATLIST, User is %USERNAME%
rem  *
:END

WIP2

@echo OFF
rem  *
rem  * ESA, Enterprise Server Administration
rem  *      Catalog List Utility, Dump Format
rem  *      The Control file is created using the Windows "echo" command
rem  *
rem  * CATGET01 - identifies the CATALOG.DAT file that will be used as input
rem  * CATTXT01 - identifies the ASCII/Text file to be used as output.
rem  * CATCTL01 - identifies the control file
rem  *
     call ..\..\Env1BASE
     set CATGET01=%BASECAT%\CATALOG.DAT
     set CATTXT01=%BASESYS%\LOGS\SYSOUT_SYS1CATDUMP.TXT
     set CATCTL01=%BASESYS%\PARMLIB\CATDUMP1.CTL
rem  *
     echo *******************************************************SYS1CATDUMP
     echo * Starting JobName ezCATDUMP, User is %USERNAME%
rem  *
     echo * Create the control file CATCTL01 to have the Catalog List
     echo * be created in a Hexadecimal Dump format.
rem  *
     if exist %CATCTL01% erase %CATCTL01%
     echo *>%CATCTL01%
     echo /DUMP>>%CATCTL01%
     echo /NOSHOWDUMP>>%CATCTL01%
     echo *>>%CATCTL01%
     echo /FLAG-HDR=Y>>%CATCTL01%
     echo /FLAG-EBC=N>>%CATCTL01%
     echo /FLAG-HEX=Y>>%CATCTL01%
     echo /FLAG-ASC=Y>>%CATCTL01%
     echo *>>CATCTL01.TXT
rem  *
     echo * Post Catalog information to a file and display to a screen
     run CBLCATC2
     if not "%ERRORLEVEL%" == "0" goto EOJNOK
     echo * The Catalog list with dump format will be displayed in a separate window...
     START NotePad %CATTXT01%
rem  *
:EOJAOK
     echo Finished JobName SYS1CATDUMP, User is %USERNAME%
     goto END
rem  *
:EOJNOK
     echo ABENDING JobName SYS1CATDUMP, User is %USERNAME%
rem  *
:END

Spread Sheet Information

The following is the column structure for an Excel spreadsheet to be used as a starting point to populate the catalog.

* HTMGENER

Column - Label

Description

A - Level Number

This should be a "1" for the primary file definition information. A zero (0) is used to define the row that is used for column headers.

B - DSN

This is a text string with a maximum length of forty-four (44) characters. It is the fully-qualified MVS Data Set Name (DSN) or VSAM Cluster Name

C - DD Name

This is a text string with a maximum length of eight (8) characters. It is the DD Name.

D - DSORG

This is the Data Set Organization.

E - RECFM

This is the Record Format.

F - lrecl-MIN

This is a numeric string with a maximum length of five (5) digits. It is the minimum or average record length.

G - lrecl-MAX

This is a numeric string with a maximum length of five (5) digits. It is the maximum record length.

H - KeyPos

This is a numeric string with a maximum length of five (5) digits. It is the key starting position. The first position in a record is 1

I - KeyLen

This is a numeric string with a maximum length of five (5) digits. It is for the length of the key.

J - VSAM Data Name

This is a text string with a maximum length of forty-four (44) characters. It is the VSAM Data Name for KSDS and ESDS

K - VSAM Index Name

This is a text string with a maximum length of forty-four (44) characters. It is the VSAM Index Name for KSDS

L - Override Job

This is a text string with a maximum length of eight (8) characters. It is the JCL Member name that will be used to override the generated JCL that is used to create the catalog entry.

M - User Job 1

This is a text string with a maximum length of eight (8) characters. It is the JCL Member name for a job that will be submitted to the internal reader after the catalog creation process is complete.

N - User Job 2

This is a text string with a maximum length of eight (8) characters. It is the JCL Member name for a job that will be submitted to the internal reader after the previous steps are completed.

O - Extended S1

This is a text string with a maximum length of four (4) characters. The content varies.
If DSORG=PO then this field contains the extension of the PDSM that is used for a dynamic PDS.

P - Extended S2

This is a text string with a maximum length of four (4) characters. The content varies.

Q - Extended M1

This is a text string with a maximum length of sixteen (16) characters. The content varies, refer to Note-1

R - Extended M2

This is a text string with a maximum length of sixteen (16) characters. The content varies, refer to Note-1

S - Extended M3

This is a text string with a maximum length of sixteen (16) characters. The content varies, refer to Note-1

T - Extended M4

This is a text string with a maximum length of sixteen (16) characters. The content varies, refer to Note-1

U - Extended X1

This is a text string with a maximum length of 256 characters. It is the path to the directory that contains the Members for a Job Override, User Job 1 and/or User Job 2.

V - Extended X2

This is a text string with a maximum length of 256 characters. The content varies.
If DSORG=PO then this field contains the path to the directory that contains the PDS Members.

Note-1:

The Extended M1 trhough M4 fields are sixteen (16) byte fields. The content varies.

If	DSORG=GDG the following values are accepted:
	LIMIT(nnn) where nnn is the number of active generations
	SCRATCH or NOSCRATCH, SCRATCH will physically delete the generation, NOSCRATCH will uncatalog the latest generation
	NOEMPTY or EMPTY, This specifies, what action is to be taken when the GDG becomes full base on the number in the LIMIT.

Format of an Excel Spread Sheet

Micro Focus Catalog Extract

WIP2

@echo OFF
rem  * *******************************************************************
rem  *             SYS1CATXPORT.cmd - a Windows Command File             *
rem  *         This program is provided by SimoTime Technologies         *
rem  *           (C) Copyright 1987-2019 All Rights Reserved             *
rem  *             Web Site URL:   http://www.simotime.com               *
rem  *                   e-mail:   helpdesk@simotime.com                 *
rem  * *******************************************************************
rem  *
rem  * Text   - Enterprise Server, Catalog Export Utility
rem  * Author - SimoTime Technologies
rem  * Date   - January 24, 1996
rem  *
rem  * LONAP  - an acronym for a List Of Names And Properties.
rem  * CSV    - an acronym for Comma-Separated-Values.
rem  * FFL    - an acronym for Fixed-Field-Length.
rem  *
rem  * INPUT... Description
rem  * CATGET01 identifies the CATALOG.DAT file.
rem  *
rem  * OUTPUT.. Description
rem  * LONAPCSV identifies the ASCII/Text file,
rem  *          records are text strings of Comma-Separated-Values (CSV).
rem  * LONAPFFL identifies the Sequential file,
rem  *          records are data strings of Fixed-Field-Length (FFL).
rem  *
     set CmdName=SYS1CATXPORT
     call ..\..\Env1BASE
     call SIMONOTE "*******************************************************%CmdName%"
     call SIMONOTE "Starting CmdName %CmdName%, User is %USERNAME% "
rem  *
rem  *             Control/Monitor
rem  *            12345678/12345678
     set SIMOCNTL=YYYNNNNN/NNNNNNNN
     set CATGET01=%BASECAT%\CATALOG.DAT
     set LONAPCSV=%BASECAT%\Wrk1\SIMOTIME.UTIL.LONAPCSV.CSV
     set LONAPFFL=%BASECAT%\Wrk1\SIMOTIME.UTIL.LONAPFFL.DAT
rem  *
     run MFCATXPT
     if not "%ERRORLEVEL%" == "0" goto EOJNOK
     call SIMONOTE "The Catalog Export Info will be displayed in a separate window..."
     START NotePad %LONAPCSV%
rem  *
:EOJAOK
     call SIMONOTE "DataTake is %CATGET01%"
     call SIMONOTE "DataMake is %LONAPCSV%"
     call SIMONOTE "DataMake is %LONAPFFL%"
     call SIMONOTE "Finished CmdName %CmdName%, User is %USERNAME% "
     goto END
rem  *
:EOJNOK
     call SIMONOTE "ABENDING CmdName %CmdName%, User is %USERNAME% "
rem  *
:END
     if "%SIMOMODE%" == "PAUSE" PAUSE

Ongoing Growth of FPMASTER.DAT

WIP2

Online Access of FPMASTER.DAT

Explore a CICS Sample Program for accessing the FPMASTER File. The FPMASTER File contains information (or properties) about the non-relational data structures used by the sample applications. The repository is a VSAM, Key-Sequenced-Data-Set (KSDS)

Build a Catalog for Micro Focus

A suite of programs is provided that will automate and document the process for creating and populating a catalog for the Micro Focus environment. The following sections provide the detailed information about the various jobs and programs.

Create Catalog Entries

Explore How to Create Catalog Entries for non-Relational Data Files and Structures and document the process.

Catalog Entries for PDS's

Explore How to Create Catalog Entries for Partioned Data Sets (PDS's) and Document the Process. This suite of scripts and documentation is for a Micro Focus Enterprise Server running on a Windows System.

Catalog Entries for GDG's

Explore How to Create Catalog Entries for Generation Data Groups (GDG's) and Document the Process.

Catalog Entries for Sequential Files

Explore How to Create Catalog Entries for Sequential Files and Document the Process. This suite of scripts and documentation is for a Micro Focus Enterprise Server running on a Windows System.

Catalog Entries for VSAM Data Sets

Explore How to Create Catalog Entries for VSAM, Key-Sequenced-Data-Sets (KSDS's) and Document the Process.

Repository Record Structure

The following (LONAPFFL.cpy) is the COBOL copy file that defines the record structure of fixed length fields for the repository of application properties or the APMASTER File that is a VSAM, key-Sequenced-Data-Set (or KSDS).

      *****************************************************************
      *               LONAPFFL.CPY - a COBOL Copy File                *
      *     Record Layout for the File Properties Repository          *
      *  The record stucture is a fixed length of 1,024 characters    *
      *         Copyright (C) 1987-2019 SimoTime Technologies         *
      *                     All Rights Reserved                       *
      *              Provided by SimoTime Technologies                *
      *        Our e-mail address is: helpdesk@simotime.com           *
      *     Also, visit our Web Site at http://www.simotime.com       *
      *****************************************************************
      *  The information that is imported into this format originates
      *  from an Excel spreadsheet. The spreadsheet is saved to an
      *  ASCII/Text File with a Comma-Separated-Values (or CSV) record
      *  structure.
      *
      *  LONAP is an acronym for a List Of Names And Properties.
      *  CSV   is an acronym for Comma-Separated-Values.
      *  FFL   is an acronym for Fixed-Field-Length.
      *
      *  A conversion program is used to convert the CSV File to a
      *  Record Sequential File with the following Fixed Field format.
      *
      *  The fields are all fixed length.
      *  Numeric fields are right-adjusted with leading zeroes.
      *  Alpha-numeric fields are left adjusted with trailing spaces.
      *
      *  Positions 73-80 of the following statements are used to
      *  identify the original columns used in the Excel spreadsheet.
      *
       01  LONAPFFL-REC.                                                Excel
           05  LONAPFFL-LEVEL-CHR.
               10  LONAPFFL-LEVEL-NBR      PIC 9(2).                    col-A
           05  LONAPFFL-FILE-INFO.
               10  LONAPFFL-DSN            PIC X(54).                   col-B
               10  LONAPFFL-DD-NAME        PIC X(17).                   col-C
               10  LONAPFFL-DSORG          PIC X(4).                    col-D
               10  LONAPFFL-RECFM          PIC X(4).                    col-E
               10  LONAPFFL-LRECL-MIN      PIC 9(5).                    col-F
               10  LONAPFFL-LRECL-MAX      PIC 9(5).                    col-G
               10  LONAPFFL-KEYPOS         PIC 9(5).                    col-H
               10  LONAPFFL-KEYLEN         PIC 9(5).                    col-I
               10  LONAPFFL-VSAM-DAT       PIC X(58).                   col-J
               10  LONAPFFL-VSAM-IDX       PIC X(58).                   col-K
           05  LONAPFFL-JOB-EXEC.
               10  LONAPFFL-JCL-OVR1       PIC X(8).                    col-L
               10  LONAPFFL-JCL-USR1       PIC X(8).                    col-M
               10  LONAPFFL-JCL-USR2       PIC X(8).                    col-N
           05  LONAP-EXTENDED-GROUP.
               10  LONAPFFL-EXTENDED-S1    PIC X(4).                    col-O
               10  LONAPFFL-EXTENDED-S2    PIC X(4).                    col-P
               10  LONAPFFL-EXTENDED-M1    PIC X(16).                   col-Q
               10  LONAPFFL-EXTENDED-M2    PIC X(16).                   col-R
               10  LONAPFFL-EXTENDED-M3    PIC X(16).                   col-S
               10  LONAPFFL-EXTENDED-M4    PIC X(16).                   col-T
               10  LONAPFFL-EXTENDED-X1    PIC X(256).                  col-U
               10  LONAPFFL-EXTENDED-X2    PIC X(256).                  col-V
           05  FILLER                      PIC X(199).
      *
      ***  LONAPFFL - End-of-Copy File - - - - - - - - - - - LONAPFFL *
      *****************************************************************
      *

Summary

This example will use a Windows System running Micro Focus Enterprise Studio to create a Key-Sequenced-Data-Set (or VSAM, KSDS) containing an inventory of Partitioned Data Sets, Generation Data Groups, Sequential Files and VSAM Data Sets. This document may nbe used as a tutorial for new programmers or as a quick reference for experienced programmers.

In the world of programming there are many ways to solve a problem. This documentation and software were developed and tested on systems that are configured for a SIMOTIME environment based on the hardware, operating systems, user requirements and security requirements. Therefore, adjustments may be needed to execute the jobs and programs when transferred to a system of a different architecture or configuration.

SIMOTIME Services has experience in moving or sharing data or application processing across a variety of systems. For additional information about SIMOTIME Services or Technologies please contact us using the information in the Contact or Feedback section of this document.

Software Agreement and Disclaimer

Permission to use, copy, modify and distribute this software, documentation or training material for any purpose requires a fee to be paid to SimoTime Technologies. Once the fee is received by SimoTime the latest version of the software, documentation or training material will be delivered and a license will be granted for use within an enterprise, provided the SimoTime copyright notice appear on all copies of the software. The SimoTime name or Logo may not be used in any advertising or publicity pertaining to the use of the software without the written permission of SimoTime Technologies.

SimoTime Technologies makes no warranty or representations about the suitability of the software, documentation or learning material for any purpose. It is provided "AS IS" without any expressed or implied warranty, including the implied warranties of merchantability, fitness for a particular purpose and non-infringement. SimoTime Technologies shall not be liable for any direct, indirect, special or consequential damages resulting from the loss of use, data or projects, whether in an action of contract or tort, arising out of or in connection with the use or performance of this software, documentation or training material.

Downloads and Links

This section includes links to documents with additional information that are beyond the scope and purpose of this document. The first group of documents may be available from a local system or via an internet connection, the second group of documents will require an internet connection.

Note: A SimoTime License is required for the items to be made available on a local system or server.

Current Server or Internet Access

The following links may be to the current server or to the Internet.

Note: The latest versions of the SimoTime Documents and Program Suites are available on the Internet and may be accessed using the icon. If a user has a SimoTime Enterprise License the Documents and Program Suites may be available on a local server and accessed using the icon.

Explore the JCL Connection for more examples of JCL functionality with programming techniques and sample code.

Explore the COBOL Connection for more examples of COBOL programming techniques and sample code.

Explore An Enterprise System Model that describes and demonstrates how Applications that were running on a Mainframe System and non-relational data that was located on the Mainframe System were copied and deployed in a Microsoft Windows environment with Micro Focus Enterprise Server.

Explore The ASCII and EBCDIC Translation Tables. These tables are provided for individuals that need to better understand the bit structures and differences of the encoding formats.

Explore The File Status Return Codes that are used to interpret the results of accessing VSAM data sets and/or QSAM files.

Internet Access Required

The following links will require an internet connect.

This suite of programs and documentation is available to download for review and evaluation purposes. Other uses will require a SimoTime Software License. Link to an Evaluation zPAK Option that includes the program members, documentation and control files.

A good place to start is The SimoTime Home Page for access to white papers, program examples and product information. This link requires an Internet Connection

Explore The Micro Focus Web Site for more information about products (including Micro Focus COBOL) and services available from Micro Focus. This link requires an Internet Connection.

Glossary of Terms

Explore the Glossary of Terms for a list of terms and definitions used in this suite of documents and white papers.

Contact or Feedback

This document was created and is maintained by SimoTime Technologies. If you have any questions, suggestions, comments or feedback please use the following contact information.

1.	Send an e-mail to our helpdesk.

1.1.	helpdesk@simotime.com.

2.	Our telephone numbers are as follows.

2.1.	1 415 763-9430 office-helpdesk

2.2.	1 415 827-7045 mobile

We appreciate hearing from you.

Company Overview

SimoTime Technologies was founded in 1987 and is a privately owned company. We specialize in the creation and deployment of business applications using new or existing technologies and services. We have a team of individuals that understand the broad range of technologies being used in today's environments. Our customers include small businesses using Internet technologies to corporations using very large mainframe systems.

Quite often, to reach larger markets or provide a higher level of service to existing customers it requires the newer Internet technologies to work in a complementary manner with existing corporate mainframe systems. We specialize in preparing applications and the associated data that are currently residing on a single platform to be distributed across a variety of platforms.

Preparing the application programs will require the transfer of source members that will be compiled and deployed on the target platform. The data will need to be transferred between the systems and may need to be converted and validated at various stages within the process. SimoTime has the technology, services and experience to assist in the application and data management tasks involved with doing business in a multi-system environment.

Whether you want to use the Internet to expand into new market segments or as a delivery vehicle for existing business functions simply give us a call or check the web site at http://www.simotime.com

Return-to-Top

Data File Information Gathering, Create and Leverage a File Properties Repository

When technology complements business

http://www.simotime.com