Data File Convert
System Information, RSEQ to LCSV
|
 |
| |
Table of Contents |
v-20.00.00 - tcntxtsb.htm |
|
|
Introduction
This document is the System Information Manual for the TCNTXT01 Test Case. This document and the links to other documents are intended to provide a greater awareness of the Data Management challenges that are encountered when sharing or moving data between systems of differing architectures. This document will focus on the process of converting multiple data files or data structures using SimoTime Technologies. The convert process includes file format, record content and field format conversion. The content of text fields may be converted between EBCDIC and ASCII. The format of numeric fields may be converted between Binary, Packed Decimal and Zoned Decimal.
This document provides information about the preparation tasks required to create a system and job environment to do the actual data file conversion tasks.
This suite of programs requires a Windows System with Micro Focus Enterprise Server and the SimoTime Library.
Explore
the SIMOTIME Web Site
for additional information about the technologies and services that are available.
We have made a significant effort to ensure the documents and software technologies are correct and accurate. We reserve the right to make changes without notice at any time. The function delivered in this version is based upon the enhancement requests from a specific group of users. The intent is to provide changes as the need arises and in a timeframe that is dependent upon the availability of resources.
Copyright © 1987-2020
SimoTime Technologies and Services
All Rights Reserved
Clarity of Field Names
The field names defined in this test case are based on Data Usage, Business Clarity and the naming guidelines for the SimoTime Test Cases.
|
1.
|
Field naming convention for Alpha-Numeric Data Strings.
|
|
1.1.
|
The default value for the COBOL USAGE Clause is "USAGE IS DISPLAY
|
|
2.
|
Field naming convention for Numeric-Values.
|
|
2.1.
|
The field name is four text strings delimited by a hypten (-) symbol.
|
|
2.1.1.
|
Common Name based on Record Structure
|
|
2.1.2.
|
Numeric Format and Field Structure are as follows
|
|
2.1.3.
|
TCNTXT01-ZDx-ii-dd
|
|
2.1.4.
|
TCNTXT01-ZDx Zoned-Decimal format
|
|
2.1.5.
|
TCNTXT01-PKx Packed-Decimal format
|
|
2.1.6.
|
TCNTXT01-BNx Binary format
|
|
2.1.7.
|
where x=S(igned) or U(nsign)
|
|
2.1.8.
|
-ii-dd ii=count of integer positions
|
|
2.1.9.
|
dd=count of decimal positions
|
|
2.2.
|
Note: For purposes of discussion the following terms may be used and should be associated with the following definitions.
|
|
2.2.1.
|
Integer - a number that is a positive, negative or zero value
|
|
2.2.2.
|
Number - or Whole Number is a number that is a positive or zero value
|
|
2.2.3.
|
Signed - an integer
|
|
2.2.4.
|
Unsign - a whole number or implied positive
|
Batch Job Scripts
A job script may be defined as a text file containing job setup information followed by job steps that identify programs to be executed along with parameters unique to the job step. A job script may be created using a text editor. The naming of a job script is determined by the Operating System. A simple job script may contain a single job step that performs a single function. A typical job script will contain multiple job steps executed in a predefined sequence. The status of each job step may be tested at the end of each job step.
|
1.1.
|
Schedule on a pre-defined basis
|
|
1.2.
|
Called from another Batch Job
|
|
1.3.
|
On demand based on a user request
|
Define Convert Requirements
The SimoTime Conversion and Validation Process is intended for conversion efforts that go beyond a simple EBCDIC to ASCII Data File Conversion. Quite often a data structure (or record structure) will require different conversion techniques to be used based on the properties of individual data strings within a data structure (or types of fields within a record).
To accomplish this the SimoTime Technologies use a technique that generates COBOL programs to do the actual file format and record content conversion at a field level of granularity. The information needed to do this is obtained from a set of user specifications (provided in a Process Control File) and the field definitions (provided in a COBOL Copy File).
The first step in the conversion effort is to create an environment (or sub-system) that is separate and isolated from other projects or processing tasks.
|
1.
|
Prepare a Conversion Environment
|
|
1.1.
|
Create a Directory Structure
|
|
1.2.
|
Populate with Job Scripts
|
|
2.
|
Create a List of PCF Names and File Specifications
|
|
2.1.
|
Determine granularity of conversioin functions
|
|
2.1.1.
|
File format conversion
|
|
2.1.2.
|
Record content conversion
|
|
2.1.3.
|
Record format conversion
|
|
2.1.3.1.
|
Single Text string
|
|
2.1.3.2.
|
Field Level Conversion
|
|
2.2.
|
Use Excel of LibreOffice Calc
|
|
3.1.
|
A single PCF Member may be created manually using a Text Editor of choice
|
|
3.2.
|
Multiple PCF Members may be created programmatically using the List of PCF Names created in the previous step.
|
1. Prepare the Environment
To preparation of a system environment starts by creating a directory (or library) structure to isolate the conversion environment and provide a location for managing the resources used for the conversion processing.
|
1.
|
Prepare a Conversion Environment
|
|
1.1.
|
Create a Directory Structure
|
|
1.2.
|
Populate with Job Scripts
|
2. Prepare a List
A list of Conversion task is manually created using spreadsheet software (Excel or LibreOffice CALC). A copy of the spreadsheet is saved as a CSV File.
|
2.
|
Create a List of PCF Names, Conversion Functions and File Specifications
|
|
2.1.
|
Determine granularity of conversioin functions
|
|
2.1.1.
|
File format conversion
|
|
2.1.2.
|
Record content conversion
|
|
2.1.3.
|
Record format conversion
|
|
2.1.3.1.
|
Single Text string
|
|
2.1.3.2.
|
Field Level Conversion
|
|
2.2.
|
Use Excel of LibreOffice Calc to make a list
|
The following shows the content of a spreadsheet that is used by the TCNTXT01 Test Case.
| | A | B | C | D | E | F | G | H | I |
J | K | L | M | N | O | P | Q | R |
S | T | U | V | W | X | Y | Z |
AA | AB | AC | AD | AE | AF | AG | AH | AI | |
| 1 | PCF Name | Status | Header Information | HTM Member | CBL Copy Member | CBL Conv Table | CBL IO Member | SYSUT1 | UT1 Org |
UT1 RecFM | UT1 Rmin | UT1 Rmax | UT1 Kpos | UT1 Klen | RecMODE | RecVARY | SYSUT2 | UT2 Org |
UT2 RecFM | UT2 Rmin | UT2 Rmax | UT2 Kpos | UT2 Klen | RecMODE | RecVARY | Conv Process |
Conv Mode 1 | Keyword 1 | Function 1 | Conv Mode 2 | Keyword 2 | Function 2/TD> | Conv Mode 3 | Keyword 3 | Function 3 | |
| 2 | TCNA2E01.pcf | A | A2E CONV, Text and Numeric Formats | tcntxtb1.htm | TCNTXTB1.cpy | AE0437B1 | | | |
| | | | | | | | | |
| | | | | | TCNA2ER1 |
A2E | &TRANSMODE | A2E | | | | | | | |
| 3 | TCNE2A01.pcf | A | E2A CONV, Text and Numeric Formats | tcntxtb1.htm | TCNTXTB1.cpy | AE0437B1 | TCNE2AC1 | | Sequential |
Fixed | 512 | 512 | | | | | | Sequential |
Fixed | 512 | 512 | | | | | TCNE2AR1 |
E2A | &TRANSMODE | E2A | | &TRANSINIT | | | | | |
| 4 | TCNEXT05.pcf | A | FLD CONV, FFL to a Text CSV Format | tcntxtb1.htm | TCNTXTB1.cpy | | TCNEXTC5 | | Sequential |
Fixed | 512 | 512 | | | | | | ASCII/Text |
Variable | 4 | 32760 | | | | | TCNEXTR5 |
EXT | &DELIMITER | , | USR | &USERDATA | CSVHDRWS.txt | USR | &USERSOJ | CSVHDRUI.txt | |
| 5 | TCNIMP05.pcf | A | FLD CONV, Text CSV to a FFL Format | tcntxtb1.htm | TCNTXTB1.cpy | | TCNIMPC5 | | ASCII/Text |
Variable | 4 | 32760 | | | | | | Sequential |
Fixed | 512 | 512 | | | | | TCNIMPR5 |
IMP | &DELIMITER | , | USR | &USERSOJ | DROP01UI.txt | | | | |
3. Prepare PCF Members
The third and final step of the preparation process is to generate or create the PCF Member.
|
3.1.
|
A single PCF Member may be created manually using a Text Editor of choice
|
|
3.2.
|
Multiple PCF Members may be created programmatically using the List of PCF Names created in the previous step.
|
The following is the job script that will create the PCF Members based on the list created in the preceding job.
@echo OFF
set JOB_NAME=MAKE_PCF_MEMBERS
rem * *******************************************************************
rem * This Job Script is provided by SimoTime Technologies *
rem * (C) Copyright 1987-2020 All Rights Reserved *
rem * Web Site URL: http://www.simotime.com *
rem * e-mail: helpdesk@simotime.com *
rem * *******************************************************************
rem *
rem * Text - Read LCSV, write RSEQ-ASC-FFL Customer Master.
rem * Author - SimoTime Technologies
rem * Date - January 24, 1996
rem *
rem * The job will execute a program that will read an ASCII/Text file
rem * that contains a list of PCF Files and specifications.
rem * Next, the program will write to a new Record Sequential (RSEQ)
rem * file that contains ASCII encoded data with fixed field length (FFL)
rem * text strings.
rem *
rem * ************
rem * * GENPCFW1 *
rem * ********cmd*
rem * *
rem * *
rem * ************
rem * * ENV1BASE *
rem * ********cmd*
rem * *
rem * *
rem * ************
rem * * SimoNOTE *
rem * ********cmd*
rem * *
rem * *
rem * ************
rem * * RUN *---------------------------*
rem * ********rts* *
rem * * ************ ************ ************
rem * * * SYSUT1 *--*--* UTGPCFC1 *-----* SYSUT2 *
rem * * * Note-1 * * ********cbl* * Note-2 *
rem * * *******rseq* * *******lseq*
rem * * *
rem * * ************ *
rem * * * SYSUT3 *--*
rem * * * Note-3 * *
rem * * *******lseq* *
rem * * *
rem * * ************ *
rem * * * SYSUT4 *--*
rem * * * Note-4 *
rem * * *******lseq*
rem * ************
rem * * EOJ *
rem * ************
rem *
rem * Note-1: SYSUT1 is a Line Sequential (or ASCII/Text file with
rem * a CSV (Comma-Separated-Values) record structure.
rem *
rem * Note-2: SYSUT2 is a Record Sequential file that contains
rem * ASCII-encoded text data in a Fixed-Field-Length (FFL)
rem * record structure.
rem *
rem * *******************************************************************
rem *
call ..\..\ENV1BASE
rem *
call SimoNOTE "*******************************************************************************%JOB_NAME% "
call SimoNOTE "* Starting JobName %JOB_NAME%, User is %USERNAME%"
call SimoNOTE "* Job_Step 01 of 03, Preparing the System and Job Environments"
set JOB_STATUS=0
set SYSOUT=%BaseLib1%\LOGS\SYSOUT_%JOB_NAME%.txt
set SYSUT1=%BaseLib1%\DATA\XLSS\GENR_PCF_LIST_NUMBERS_CONV.csv
set SYSUT3=%BaseLib1%\PARMLIB\GENPCFT1.txt
set PGMFLAGS=YNNNNNNN/NNNNNNNN
set PCF_LIBR=%BaseLib1%\DATA\WRK1\
set GEN_LIBR=%BaseLib1%\DATA\GENS
rem *
call SimoNOTE "* --------------------------------------------------------------------------- *"
call SimoNOTE "* Job_Step 02 of 03, Generate a PCF Member"
call SimoNOTE "* DataTAKE SYSUT1=%SYSUT1% "
call SimoNOTE "* DataPREP SYSUT3=%SYSUT3% "
run UTGPCFC5
if not "%ERRORLEVEL%" == "0" set JOB_STATUS=10
if not "%JOB_STATUS%" == "0" goto EOJTAG
goto EOJTAG
rem *
:EOJTAG
call SimoNOTE "*-----------------------------------------------------------------------------*"
call SimoNOTE "* JOB_STEP 03 of 03, End of Job Processing"
call SimoNOTE "* SIMONOTE Job Log is %SIMONOTE% "
if "%JOB_STATUS%" == "0" goto EOJAOK
:EOJNOK
call SimoNOTE "* ABENDING JOB_STATUS=%JOB_STATUS% JobName %JOB_NAME% "
goto :EOJEND
:EOJAOK
call SimoNOTE "* Finished JOB_STATUS=%JOB_STATUS% JobName %JOB_NAME% "
goto :EOJEND
:EOJEND
call SimoNOTE "* Conclude SYSOUT is %SYSOUT% "
if not "%SIMOGENS%" == "BATCH" pause
exit /B %JOB_STATUS%
Create Convert Programs
The SIMOTIME Technologies provides a data file conversion process that uses generated COBOL programs to do the actual file format and record content conversion. The COBOL source code that is generated is ANSI 2002 compliant and may be compiled and executed on an IBM Mainframe System, a Windows System, a Linux System or a UNIX System.
For this test case the conversion programs were created, compiled and promoted on a Windows System using SIMOTIME Technologies and Micro Focus Visual COBOL. The following provides additional details.
|
1.
|
Generate COBOL Conversion Programs
|
|
1.1.
|
Single Conversion Program
|
|
1.2.
|
Multiple Conversion Programs
|
|
2.
|
Compile COBOL Conversion Programs
|
|
2.1.
|
Execute a batch job script that will compile a single program
|
|
2.2.
|
Execute a batch job script that will compile multiple programs
|
|
3.
|
Promote COBOL Conversion Programs
|
|
3.1.
|
Copy the Load (or executable) members to the LOADLIB
|
|
3.2.
|
Execute a batch job script that will do the data file conversion.
|
1. Generate Convert Programs
This test case will use a Job Script to generate the COBOL source cade that wil be used to do a data file conversion. This task will be accomplished by reading a Process Control File (PCF) as the primary input. If the Data File Conversion required a field level conversion then the field defdinitions from a COBOL Copy file will be used.
|
1.
|
Generate COBOL Conversion Programs
|
|
1.1.
|
Single Conversion Program
|
|
1.2.
|
Multiple Conversion Programs
|
The following is the job script that will generate the COBOL Source Code for the conversion programs.
@echo OFF
set CmdName=ADM_GENRONE_CONVERTS
rem * *******************************************************************
rem * This Job Script is provided by SimoTime Technologies *
rem * (C) Copyright 1987-2020 All Rights Reserved *
rem * Web Site URL: http://www.simotime.com *
rem * e-mail: helpdesk@simotime.com *
rem * *******************************************************************
rem * This procedure calls the following Windows Command Files.
rem * 1. ENV1BASE.cmd - provide a single job script for preparing
rem * the System environment.
rem * 2. USERCONV.cmd - provide a single job script for preparing
rem * the Job environment.
rem * 3. SIMONOTE.cmd - provide a consistent process for displaying
rem * messages to the screen and writing to a journal
rem * or log file.
rem * 4. ZPCF2CVT.cmd - do the generation of the COBOL programs that
rem * will be used for the file conversion process.
rem * ************************************************************************
rem * The objective of this batch job is to generate the COBOL program(s)
rem * required to do a file format or record content conversion of a single
rem * data file.
rem * For more information about the data conversion utility process refer
rem * to the following.
rem *
rem * http://www.simotime.com/utconv01.htm
rem *
rem * The following two call statements will set the environment variables.
rem * The call to ENV1BASE will set the System or Global/Shared variables.
rem * The call to USERCONV will set the Job-specific variables.
rem *
call ..\..\ENV1BASE
call USERCONV
rem *
call SimoNOTE "*******************************************************************************%CmdName%.cmd"
call SimoNOTE "* Step 01 of 04, Prepare the Job Environment."
rem *
rem * Set the variables for the success/failure counters.
set AOK_Count=0
set NOK_Count=0
rem *
rem * Do not drop (or delete) the interim _genr and _spec files
rem * and pause at the end of job.
rem *
set SIMODROP=N
set SIMOGENS=PAUSE
set GENILIBR=%BASELIB1%\SIMOGENS\CONVERTS\INCLUDES
set SIMOCFG1=%BASELIB1%\SIMOGENS\CONVERTS\SIMOCFG1.cfg
set SWITCHES=NNNNNNNN/NNNNNNNN
rem *
call SimoNOTE "* --------------------------------------------------------------------------- *
call SimoNOTE "* Step 02 of 04, Define PCF File Name in Separate Test Editor Window,"
call SimoNOTE "* validate or change name then close Text Editor to continue..."
set MBR_LIST=%BASELIB1%\SIMOGENS\CONVERTS\SIMOTIME.LISTONE.PCF4CONV.lst
if "%SimoGENS%" == "PAUSE" notepad %MBR_LIST%
rem *
call SimoNOTE "* --------------------------------------------------------------------------- *
call SimoNOTE "* Step 03 of 04, Generate the Convert Programs."
for /F "eol=; tokens=1,2* delims=." %%i in (%MBR_LIST%) do CALL ZPCF2CVT %%i
rem *
:EOJTAG
call SimoNOTE "* --------------------------------------------------------------------------- *
call SimoNOTE "* Step 04 of 04, End of Job Processing."
call SIMONOTE "* AOKcount Compile Count for AOK is %AOK_Count% "
call SIMONOTE "* NOKcount Compile Count for NOK is %NOK_Count% "
if "%NOK_Count%" == "0" (call SIMONOTE "* Finished JobName %CmdName%.cmd "
goto :EOJEND)
call SIMONOTE "* ABENDING JobName %CmdName%.cmd "
:EOJEND
if not "%SimoGENS%" == "BATCH" pause
2. Compile Convert Programs
This test case will use a Job Script to Compile the generated COBOL Source Code and create an executable member.
|
2.
|
Compile COBOL Conversion Programs
|
|
2.1.
|
Execute a batch job script that will compile a single program
|
|
2.2.
|
Execute a batch job script that will compile a multiple programs
|
The following is the job script that will compile the COBOL Source Code using Micro Focus COBOL.
@echo OFF
set CMD_NAME=CBUT2AE1
rem * *******************************************************************
rem * This Job Script is provided by SimoTime Technologies *
rem * (C) Copyright 1987-2020 All Rights Reserved *
rem * Web Site URL: http://www.simotime.com *
rem * e-mail: helpdesk@simotime.com *
rem * *******************************************************************
rem * This procedure will create a file containing a list of all the
rem * COBOL (i.e. filename.cbl) programs. This list will then be used to
rem * compile all the programs in the COBOLUT2 Folder (or Library).
rem *
rem * This procedure calls the following Windows Command Files.
rem * 1. ENV1BASE.CMD - provides a single point for setting commonly
rem * used environment variables.
rem * 2. SIMONOTE.CMD - provides a consistent process for displaying
rem * messages to the screen and writing to a journal
rem * or log file.
rem * 3. ZCBL2GNT.CMD - this will do the actual compile of the specified
rem * program.
rem *
rem * Note: To specify the directory to be used for the IDY files use
rem * the COBIDY Compiler Directive and set the COBIDY environment
rem * variable.
rem * ************************************************************************
setlocal
call ..\ENV1BASE
rem *
call SIMONOTE "*******************************************************************************%CMD_NAME%.CMD"
call SIMONOTE "* Starting JobName %CMD_NAME%.CMD"
call SIMONOTE "* Job_Step 01 of 03, Preparing the System and Job Environments"
rem *
rem * Set the environment variable to use the MiFoAsc.DIR directives file
rem * as the default.
set DIRSLIB1=%BaseLib1%\DIRS
set zDIRS=%BaseLib1%\DIRS\MiFoAscIBMcomp.DIR
rem * The following two statements will set the environment variables to
rem * a "Y" for Yes or "N" for No. A "Y" will cause the function to be
rem * performed by the called procedure. An "N" value will not perform the
rem * function.
set EraseAfter=N
set TransferGNT=N
rem *
rem * Read the file containing a list of program names and pass the contents
rem * of each record to the command file that does the actual compile.
rem * Each record in the file contains an eight character program name.
set AOK_Count=0
set NOK_Count=0
set SLIB1=COBOLUT2
set TLIB1=HOLD\UT2A
rem *
call SIMONOTE "* --------------------------------------------------------------------------- *"
call SIMONOTE "* Job_Step 02 of 03, Compile the COBOL Programs..."
set MBR_LIST=%BASELIB1%\ADM1\CNTL\SIMOTIME.ADM1.COBOLUT2.lst
call SIMONOTE "* MBR_LIST is %MBR_LIST% "
dir /on /b /a-d %BaseLib1%\%SLIB1%\*.CBL>%MBR_LIST%
for /F "eol=; tokens=1,2* delims=." %%i in (%MBR_LIST%) do CALL zCBL2GNT %%i %SLIB1% %TLIB1%
rem *
:EOJTAG
call SIMONOTE "* --------------------------------------------------------------------------- *"
call SIMONOTE "* Job_Step 03 of 03, End of Job Processing..."
call SIMONOTE "* AOKcount Compile Count for AOK is %AOK_Count% "
call SIMONOTE "* NOKcount Compile Count for NOK is %NOK_Count% "
if not "%NOK_Count%" == "0" goto EOJNOK
:EOJAOK
call SIMONOTE "* Finished JobName %CMD_NAME%.CMD "
goto END
:EOJNOK
call SIMONOTE "* ABENDING JobName %CMD_NAME%.CMD "
:END
if not "%SimoGENS%" == "BATCH" pause
endlocal
3. Promote Convert Programs
This test case will use a Job Script to tranfer the executable members from a holding library to the LOADLIB.
|
3.
|
Promote COBOL Conversion Programs
|
|
3.1.
|
Copy the Load (or executable) members to the LOADLIB
|
|
3.2.
|
Execute a batch job script to that will do the data file conversion.
|
The following is the job script that will promote the executable conversion programs to the LOADLIB.
@echo OFF
set JOB_NAME=ADM1_XFER_GNTS_UT2A_TO_LOADLIB
rem * *******************************************************************
rem * This Job Script is provided by SimoTime Technologies *
rem * (C) Copyright 1987-2020 All Rights Reserved *
rem * Web Site URL: http://www.simotime.com *
rem * e-mail: helpdesk@simotime.com *
rem * *******************************************************************
rem * This job script will copy user-defined files from one folder to
rem * another based on a list of file names.
rem *
rem * This batch job script calls the following Windows Command Files.
rem * 1. ENV1BASE.cmd - provides a single point for setting commonly
rem * used environment variables.
rem * 2. SIMONOTE.cmd - provides a consistent process for displaying
rem * messages to the screen and writing to a journal
rem * or log file.
rem * 3. ZSYS1_COPY_FILE.cmd - this will do the actual copy of the files.
rem * *******************************************************************
call ..\ENV1Base
rem *
call SIMONOTE "*******************************************************************************%JOB_NAME%.cmd"
call SIMONOTE "* Starting JOB_NAME %JOB_NAME%.cmd"
call SIMONOTE "* Job_Step 01 of 03, Preparing the System and Job Environments"
rem *
set AOK_Count=0
set NOK_Count=0
set SourceLib=%BASELIB1%\HOLD\UT2A
set TargetLib=%BASELIB1%\LOADLIB
rem *
call SIMONOTE "* --------------------------------------------------------------------------- *"
call SIMONOTE "* Job_Step 02 of 03, Generate a List and Copy Executable Members..."
set MBR_LIST=%BASELIB1%\ADM1\CNTL\SIMOTIME.ADM1GNTS.COBOLUT2.lst
dir /on /b /a-d %SourceLib%\*.gnt>%MBR_LIST%
call SIMONOTE "* MBR_LIST is %MBR_LIST% "
for /F "eol=; tokens=1,2* delims=." %%i in (%MBR_LIST%) do call ZSYS1_COPY_FILE %SourceLib%\%%i.gnt %TargetLib%\%%i.gnt
rem *
:EOJTAG
call SIMONOTE "* --------------------------------------------------------------------------- *"
call SIMONOTE "* Job_Step 03 of 03, End of Job Processing..."
call SIMONOTE "* AOKcount File Copy Count for AOK is %AOK_Count% "
call SIMONOTE "* NOKcount File Copy Count for NOK is %NOK_Count% "
if not "%NOK_Count%" == "0" goto EOJNOK
:EOJAOK
call SIMONOTE "* Finished JobName %JOB_NAME%.CMD "
goto END
:EOJNOK
call SIMONOTE "* ABENDING JobName %JOB_NAME%.CMD "
:END
if not "%SimoGENS%" == "BATCH" pause
endlocal
Execute the Convert Programs
The following will provide links to the Run Book for the TCNTXT01 Test Case.
|
1.
|
Execute the Convert Programs
|
|
1.1.
|
Link to Run Book via Local System
|
|
1.2.
|
Link to Run Book via SimoTime Web Site
|
The Review Process
In today's environments there are a variety of data validation alternatives. This section will focus on three popular techniques. The 1st technique is Human Observation. The 2nd is a Data File Compare. The 3rd is the accumulation and review of Summary Totals.
Hex-Dump Review
The following Process Control File (PCF) (TCNHXR01.pcf) contains the specifications that will be used to generate the programs that will be used to provide the hexadecimal dump of user-defined records contained in the Record Sequential File that is used with this test case.
***********************************************************************
* TCNHXR01.pcf - a Process Control File *
* Hex-Dump for User-Defined records in a Record Sequential File *
* (C) Copyright 1987-2020 All Rights Reserved *
* Web Site URL: http://www.simotime.com *
* e-mail: helpdesk@simotime.com *
***********************************************************************
* SYSUT1 A Record Sequential (RSEQ) File, RSEQ, 512-byte records. *
* SYSUT2 A Line Sequential (LSEQ) file, variable-length records. *
* SYSUT3 A Line Sequential (LSEQ) file, variable-length records. *
***********************************************************************
* This Process Control File will be used to generate a COBOL program
* that will do a hexadecimal dump of user-defined records contained
* in the RSEQ File.
*
* Refer to http://www.simotime.com/utcomp01.htm for additional detail
* about the PCF statements for Data Validate, Dump and Compare.
*
* The following group of statements will define the high level
* functions and processes to be performed.
*
&SIMOPREP call ..\ENV1BASE
&USERPREP call USERCOGI
&CONFORM IBM
&USRMODEL SYSHEXS3.txt
&USRXFILE AE0437B1
*
* The following group of statements will define the behavioral
* characteristics and environment variable for the file I/O Program
* to be generated.
*
*HEAD34 ....:....1....:....2....:....3....
&HEAD34 HEX-Dump from RSEQ_512 to LSEQ_512
&PROGID TCNHXRC1
&UT1ACCESS SEQUENTIAL
&SYSUT1 name=SYSUT1 org=Sequential recfm=fixed rlen=512
&SYSUT2 name=SYSUT2 org=ASCII/Text recfm=Variable rlen=512
&SYSUT3 name=SYSUT3 org=ASCII/Text recfm=Variable rlen=80
*
* The following will determine the HEX-Dump output. The &DFORMAT
* statement will create five (5) lines of dump information.
* ASC - 1 line of possible ASCII character translation.
* HEX - 2 lines of HEX dump. 1st line is high-order nibble.
* 2nd line is low-order nibble.
* EBC - 1 line of possible EBCDIC character translation.
* RPI - 1 line for Relative Position Indicator.
*
&HEXDUMP SYSUT1 pos 0001 len 512
&DFORMAT ASC HEX EBC RPI
&SYSLOG ENABLED
*
&END
Note: Since the generated program is not sensitive to the record content the program may be used to dump the content of any record sequential file with a fixed-length record structure of 512 bytes.
File Compare by Logical Record
The following Process Control File (PCF) (TCNVAL01.pcf) contains the specifications that will be used to generate the programs that do the actual file compare.
***********************************************************************
* This is an example of the compare specifications to generate a *
* Data File Comparison Program. This is used by SimoZAPS *
* SimoTime Technologies *
* (C) Copyright 1987-2020 All Rights Reserved *
* Web Site URL: http://www.simotime.com *
* e-mail: helpdesk@simotime.com *
***********************************************************************
*
* The following group of statements will define the high level
* functions and processes to be performed.
*
&SIMOPREP call ..\..\ENV1BASE
&USERPREP call USERCOMP
&FUNCTION COMPARE
&CONFORM IBM
&USRMODEL SYSCOMP5.txt
&TBLAECOMP HEXTABLE
*
* The following group of statements will define the behavioral
* characteristics and environment variables for the file I/O
* functions within the Program to be generated.
*
*HEAD34 ....:....1....:....2....:....3....
&HEAD34 Data File Compare for RSEQ_512
&progid TCNVALC1
&SYSUT1 org=SEQUENTIAL recfm=FIXED rlen=512
&SYSUT2 org=SEQUENTIAL recfm=FIXED rlen=512
*
* The following group of statements will define the behavioral
* characteristics and environment variables for the compare functions
* within the Program to be generated.
*
&COMPARE PHYSICAL SYSUT1 pos 1 len 512 SYSUT2 pos 1 len 512
*
&IFNECODE 4
&DELTAMAX 10 EOF
*
&DFORMAT ASC HEX EBC RPI
&DSTRING 1 512
&SYSLOG SIMOL32K HTML
&SYSOUT DISABLED
*
&END
Validate Numeric Integrity
The following Process Control File (PCF) (TCNTOT01.pcf) contains the specifications that will be used to generate a set of COBOL programs that will accumulate totals for user-defined numeric fields and provide a record count.
***********************************************************************
* TCNTOT01.pcf - a Process Control File *
* Accumulate Record Counts and Totals for Numeric Fields *
* (C) Copyright 1987-2019 All Rights Reserved *
* Web Site URL: http://www.simotime.com *
* e-mail: helpdesk@simotime.com *
***********************************************************************
* SYSUT1 is a Record Sequential File with 512-byte records.
***********************************************************************
* This Process Control File will be used to generate a set of COBOL
* programs that will accumulate totals for user-defined numeric fields
* and provide a record count.
*
* Refer to http://www.simotime.com/utcomp01.htm for additional detail
* about the PCF statements for Data Validate, Dump and Compare.
*
* The following group of statements will define the high level
* functions and processes to be performed.
*
&SIMOPREP call ..\Env1BASE
&USERPREP call USERCOGI
&CONFORM IBM
©FILE TCNTXTB1.cpy
&HTMLFILE tcntxtb1.htm
&USRMODEL SYSNTOT1.txt
*
* The following group of statements will define the behavioral
* characteristics and environment variable for the file I/O Program
* to be generated.
*
*HEAD34 ....:....1....:....2....:....3....
&HEAD34 Accumulate Totals for Numeric Data
&PROGID TCNTOTC1
&SYSUT1 name=TCNTOTD1 org=Sequential recfm=FIXED rmax=512
*
* The following two statements are used when the records in the data
* file (input or output) are a Comma-Separated-Values (or CSV) format.
*
&DELIMITER ,
&FRAME "
*
* The following group of statements will define the behavioral
* characteristics and environment variables for the record content
* conversion program to be generated.
* The following defines the name of the callable routine for special
* numeric processing.
*
&NUMCALL TCNTOTR1
&NUMREC TCNTXT01-RECORD
*
* The following group of statements will define the type of processing
* and the name of the numeric fields.
*
&UT1TOTAL TCNTXT01-ZDS-09-00
&UT1TOTAL TCNTXT01-PKS-00-10
&UT1TOTAL TCNTXT01-PKS-00-07
&UT1TOTAL TCNTXT01-PKS-01-07
&UT1TOTAL TCNTXT01-PKS-07-02
&UT1TOTAL TCNTXT01-BNS-00-10
&UT1TOTAL TCNTXT01-BNS-00-07
&UT1TOTAL TCNTXT01-BNS-01-07
&UT1TOTAL TCNTXT01-BNS-07-02
*
&END
Technical Details
This section provides additional technical details about the modules that are included in this test case.
Record Structure, PCF Names and Specifications
*****************************************************************
* UTPLSTB1.cpy - a COBOL Copy File *
* Record Structure for List of PCF Files *
* Copyright (C) 1987-2020 SimoTime Technologies *
* All Rights Reserved *
* Provided by SimoTime Technologies *
* Our e-mail address is: helpdesk@simotime.com *
* Also, visit our Web Site at http://www.simotime.com *
*****************************************************************
* Define the record structure of a file that contains the
* list of PCF file names and specifications.
*
* The notation in columns 73-80 is for documentation purposes
* only and identifies the column that would be used for the
* field content if a spreadsheet software package were used
* to create test data.
*
* .1....:....2....:....3....:....4....:....5....:....6....:....7....:....8
*
01 PCF-LIST-RECORD.
05 PCF-NAME pic X(12). COL-A
05 PCF-STATUS pic X. COL-B
05 PCF-HEADER-INFO pic X(34). COL-C
05 PCF-HTM-MEMBER pic X(12). COL-D
05 PCF-CBL-COPY-MEMBER pic X(12). COL-E
05 PCF-CBL-CONV-TABLE pic X(12). COL-F
05 PCF-CBL-IO-MEMBER pic X(12). COL-G
05 PCF-SYSUT1 pic X(12). COL-H
05 PCF-SYSUT1-ORG pic X(12). COL-I
05 PCF-SYSUT1-RECFM pic X(12). COL-J
05 PCF-SYSUT1-RMIN pic 9(5). COL-K
05 PCF-SYSUT1-RMAX pic 9(5). COL-L
05 PCF-SYSUT1-KPOS pic 9(5). COL-M
05 PCF-SYSUT1-KLEN pic 9(5). COL-N
05 PCF-SYSUT1-RECMODE pic X(7). COL-O
05 PCF-SYSUT1-RECVARY pic X(7). COL-P
05 PCF-SYSUT2 pic X(12). COL-Q
05 PCF-SYSUT2-ORG pic X(12). COL-R
05 PCF-SYSUT2-RECFM pic X(12). COL-R
05 PCF-SYSUT2-RMIN pic 9(5). COL-T
05 PCF-SYSUT2-RMAX pic 9(5). COL-U
05 PCF-SYSUT2-KPOS pic 9(5). COL-V
05 PCF-SYSUT2-KLEN pic 9(5). COL-W
05 PCF-SYSUT2-RECMODE pic X(7). COL-X
05 PCF-SYSUT2-RECVARY pic X(7). COL-Y
05 PCF-CONV-PROCESS pic X(50). COL-Z
05 PCF-CBL-CONV-MODE-1 pic X(3). COL-AA
05 PCF-KEYWORD-1 pic X(11). COL-AB
05 PCF-FUNCTION-1 pic X(80). COL-AC
05 PCF-CBL-CONV-MODE-2 pic X(3). COL-AD
05 PCF-KEYWORD-2 pic X(11). COL-AE
05 PCF-FUNCTION-2 pic X(80). COL-AF
05 PCF-CBL-CONV-MODE-3 pic X(3). COL-AG
05 PCF-KEYWORD-3 pic X(11). COL-AH
05 PCF-FUNCTION-3 pic X(80). COL-AI
05 PCF-FILLER pic X(457). COL-AJ
*
*** UTPLSTB1 - End-of-Copy File - - - - - - - - - - - UTPLSTB1 *
*****************************************************************
*
Job Scripts
A job script may be defined as a text file containing job setup information followed by job steps that identify programs to be executed along with parameters unique to the job step. A job script may be created using a text editor. The naming of a job script is determined by the Operating System and user-defined naming conventions. A simple job script may contain a single job step that performs a single function. A typical job script will contain multiple job steps executed in a predefined sequence. The status of each job step may be tested at the end of each job step.
The structure of the Windows Job Scripts used in this test case start with the following two statements
@echo OFF
set JOB_NAME=name-of-job-script
The first statement will reduce the number of messages displayed to the system console. The second statement identifies the job name.
Typically, each job script will contain a minimum of three (3) job steps. The first job step will prepare the system and job environments. The second job step will execute the user-defined function. The third job step will do the End-of-Job processing.
The job scripts will access (or call) two additional job scripts. The ENV1BASE job script is used as a single point of reference to set commonly used (or shared)) environment variables. The SIMONOTE job script is called to post messages to the system console and a job log file in a consistent manner. Additional information about these two job scripts may be found at the following link.
Explore How to use Common or Shared Routines that perform repetitive tasks in a consistent manner. This link will provide information about setting common environment variables and many other utilitarian tasks.
|
The following is an overview of the Job Scripts for a Microsoft Windows System |
|
1.
|
TCNE2AW1 - Read EBC Record Sequential, write ASC Record Sequential.
|
|
1.1.
|
Read an EBCDIC-encoded, record sequential file containing numeric values that are stored in a Zoned-Decimal, Packed-Decimal and Binary format.
|
|
1.2.
|
Call TCNE2AR1 to convert text fields from EBCDIC to ASCII encoding. Maintain numeric integrity and format for the Zoned-Decimal, Packed-Decimal and Binary fields.
|
|
1.3.
|
Create and write to a new ASCII-encoded, record sequential file.
|
|
2.
|
TCNEXTW5 - Read ASC RSEQ file containing Packed-Decimal, Make LSEQ_CSV.
|
|
2.1.
|
Read an ASCII-encoded, record sequential file containing numeric values that are stored in a Zoned-Decimal, Packed-Decimal and Binary format.
|
|
2.2.
|
Call TCNEXTR5 to convert the record structure and content. Numeric formats for the Zoned-Decimal, Packed-Decimal and Binary fields will be expanded to a text-oriented format with a separate sign byte and an explicit decimal point.
|
|
2.3.
|
Create and write to a new ASCII-encoded, line sequential file with a Comma-Separated Values (CSV) record structure.
|
|
3.
|
TCNHXDW1 - Create a Hex-Dump (HTML Format) of a Record Sequential file.
|
|
3.1.
|
Read an ASCII-encoded, record sequential file containing numeric values that are stored in a Zoned-Decimal, Packed-Decimal and Binary format.
|
|
3.2.
|
Create the Hexadecimal-Dump Text/HTML strings.
|
|
3.3.
|
Create and write the Hex-Dump Information to a new ASCII-encoded, line sequential file.
|
|
3.4.
|
Review the Hex-Dump file with a browser of choice.
|
|
4.
|
TCNHXDW8 - Create a Hex-Dump (HTML Format) of a file containing 512-byte records.
|
|
4.1.
|
Read an EBCDIC-encoded, record sequential file containing numeric values that are stored in a Zoned-Decimal and Packed-Decimal and Binary format.
|
|
4.2.
|
Create the Hexadecimal-Dump Text/HTML strings.
|
|
4.3.
|
Create and write the Hex-Dump Information to a new ASCII-encoded, line sequential file.
|
|
4.4.
|
Review the Hex-Dump file with a browser of choice.
|
|
5.
|
TCNHXRW1 - Create a Hex-Dump (TEXT Format) of a Record Sequential file.
|
|
5.1.
|
Read an ASCII-encoded, record sequential file containing numeric values that are stored in a Zoned-Decimal, Packed-Decimal and Binary format.
|
|
5.2.
|
Create the Hexadecimal-Dump text strings.
|
|
5.3.
|
Create and write the Hex-Dump Information to a new ASCII-encoded, line sequential file.
|
|
5.4.
|
Review the Hex-Dump file with a text editor of choice.
|
|
6.
|
TCNHXRW8 - Create a Hex-Dump (TEXT Format) of a Record Sequential file.
|
|
6.1.
|
Read an EBCDIC-encoded, record sequential file containing numeric values that are stored in a Zoned-Decimal and Packed-Decimal and Binary format.
|
|
6.2.
|
Create the Hexadecimal-Dump text strings.
|
|
6.3.
|
Create and write the Hex-Dump Information to a new ASCII-encoded, line sequential file.
|
|
6.4.
|
Review the Hex-Dump file with a text editor of choice.
|
|
7.
|
TCNIMPW5 - Read LSEQ_CSV, Make RSEQ containing Packed-Decimal.
|
|
8.
|
TCNIMPW8 - Read LSEQ_CSV, Make RSEQ containing Packed-Decimal.
|
|
9.
|
TCNPADW1 - Start Notepad, review CSV File.
|
|
10.
|
TCNTOTW1 - Read a file, Calculate record Count and numeric totals.
|
|
11.
|
TCNTXTW8 - Read LSEQ ASCII File, create RSEQ EBCDIC File.
|
|
11.1.
|
Read an ASCII-encoded, record sequential file containing numeric values that are stored in a Zoned-Decimal, Packed-Decimal and Binary format.
|
|
11.2.
|
Create the Hexadecimal-Dump text strings.
|
|
11.3.
|
Create and write the Text Strings, Numeric Values and Hex-Dump Information to a new ASCII-encoded, line sequential file that is defined as the SYSOUT device.
|
|
11.4.
|
Review the Hex-Dump file with a text editor of choice.
|
|
12.
|
TCNUSRW1 - User Review of record content within a Sequential File.
|
|
13.
|
TCNVALW1 - Compare two Record Sequential Files.
|
|
13.1.
|
Read a record from each input file.
|
|
13.2.
|
Compare the full content of the two records and the post results.
|
|
13.3.
|
Repeat until end of file.
|
Summary
This document and the links to other documents are intended to provide a greater awareness of the Data Management challenges that are encountered when sharing, moving or converting data.
In the world of programming there are many ways to solve a problem. This documentation and software were developed and tested on systems that are configured for a SIMOTIME environment based on the hardware, operating systems, user requirements and security requirements. Therefore, adjustments may be needed to execute the jobs and programs when transferred to a system of a different architecture or configuration.
SIMOTIME Services has experience in moving or sharing data or application processing across a variety of systems. For additional information about SIMOTIME Services or Technologies please contact us using the information in the Comments or Feedback section of this document.
Software Agreement and Disclaimer
Permission to use, copy, modify and distribute this software, documentation or training material for any purpose requires a fee to be paid to SimoTime Technologies. Once the fee is received by SimoTime the latest version of the software, documentation or training material will be delivered and a license will be granted for use within an enterprise, provided the SimoTime copyright notice appear on all copies of the software. The SimoTime name or Logo may not be used in any advertising or publicity pertaining to the use of the software without the written permission of SimoTime Technologies.
SimoTime Technologies makes no warranty or representations about the suitability of the software, documentation or learning material for any purpose. It is provided "AS IS" without any expressed or implied warranty, including the implied warranties of merchantability, fitness for a particular purpose and non-infringement. SimoTime Technologies shall not be liable for any direct, indirect, special or consequential damages resulting from the loss of use, data or projects, whether in an action of contract or tort, arising out of or in connection with the use or performance of this software, documentation or training material.
Downloads and Links
This section includes links to documents with additional information that are beyond the scope and purpose of this document. The first group of documents may be available from a local system or via an internet connection, the second group of documents will require an internet connection.
Note: A SimoTime License is required for the items to be made available on a local system or server.
Current Server or Internet Access
The following links may be to the current server or to the Internet.
This is the Run Book for the TCNTXT01 Test Case. This test case will convert the file format, record content and record structure of an EBCDIC-encoded file that was created and transferred from an IBM Mainframe System to a format and structure that may be easily imported and accessed from within an ASCII-oriented relational data base or a spread sheet using Excel or LibreOffice Calc. This expanded request will involve more than a simple EBCDIC to ASCII conversion. This test case will describe and demonstrate how to identify and convert text strings between EBCDIC and ASCII and how to identify, expand and convert numeric values that are formatted as Signed-Zoned-Decimal, Packed-Decimal or Binary.
Explore the non-Relational Data Connection for more examples of accessing methodologies and coding techniques for Data Files and VSAM Data Sets.
Explore The ASCII and EBCDIC Translation Tables. These tables are provided for individuals that need to better understand the bit structures and differences of the encoding formats.
Explore The File Status Return Codes to interpret the results of accessing VSAM data sets and/or QSAM files.
Explore How to use Common or Shared Routines that perform repetitive tasks in a consistent manner. This link will provide information about setting common environment variables and many other utilitarian tasks.
Internet Access Required
The following links will require an internet connect.
A good place to start is
The SimoTime Home Page
for access to white papers, program examples and product information. This link requires an Internet Connection
Explore
The Micro Focus Web Site
for more information about products and services available from Micro Focus. This link requires an Internet Connection.
Glossary of Terms
Explore the Glossary of Terms for a list of terms and definitions used in this suite of documents and white papers.
Comments or Feedback
This document was created and is maintained by SimoTime Technologies. If you have any questions, suggestions, comments or feedback please use the following contact information.
|
1.
|
Send an e-mail to our helpdesk.
|
|
2.
|
Our telephone numbers are as follows.
|
|
2.1.
|
1 415 763-9430 office-helpdesk
|
|
2.2.
|
1 415 827-7045 mobile
|
We appreciate hearing from you.
Company Overview
SimoTime Technologies was founded in 1987 and is a privately owned company. We specialize in the creation and deployment of business applications using new or existing technologies and services. We have a team of individuals that understand the broad range of technologies being used in today's environments. Our customers include small businesses using Internet technologies to corporations using very large mainframe systems.
Quite often, to reach larger markets or provide a higher level of service to existing customers it requires the newer Internet technologies to work in a complementary manner with existing corporate mainframe systems. We specialize in preparing applications and the associated data that are currently residing on a single platform to be distributed across a variety of platforms.
Preparing the application programs will require the transfer of source members that will be compiled and deployed on the target platform. The data will need to be transferred between the systems and may need to be converted and validated at various stages within the process. SimoTime has the technology, services and experience to assist in the application and data management tasks involved with doing business in a multi-system environment.
Whether you want to use the Internet to expand into new market segments or as a delivery vehicle for existing business functions simply give us a call or check the web site at http://www.simotime.com
Copyright © 1987-2020
SimoTime Technologies and Services
All Rights Reserved |
| When technology complements business |
| http://www.simotime.com |