Data File Conversion Self-Study or Classroom

Introduction

This session will discuss the process and associated risk of a data migration that includes a conversion of the file formats (i.e. Mainframe to Micro Focus) or file content (i.e. EBCDIC to ASCII).

Included in the session is a discussion of the possible issues that may occur and corrective actions that may be required during the data conversion process of an application migration effort. An exercise in the form of a sample Proof-of-Concept (POC) is provided that will use various technologies to convert a typical mainframe, EBCDIC-encoded data file to a Micro Focus file format with ASCII-encoded records that may be processed on a Windows platform.

To perform this exercise requires a basic knowledge of the Windows environment and Micro Focus Net Express. Additional help is provided for individuals that may not be proficient in the use of Net Express or have forgotten how to perform a particular function.

We have made a significant effort to ensure the documents and software technologies are correct and accurate. We reserve the right to make changes without notice at any time. The function delivered in this version is based upon the enhancement requests from a specific group of users. The intent is to provide changes as the need arises and in a timeframe that is dependent upon the availability of resources.

Copyright © 1987-2019
SimoTime Technologies and Services
All Rights Reserved

Objectives of Session

The objective of this sample Proof-of-Concept POC is to define a practical process for converting between EBCDIC encoded file formats used on the Mainframe and ASCII encoded file formats used by Windows or UNIX. The following is a list of the minimum tasks to be completed for this Proof of Concept.

Requirements

If this session is being performed through self-study then it will be necessary to install the following software on a Windows system. Windows/XP or Windows Server is preferred.

Materials Provided

For the classroom environment two directories (c:\DataMig1 and c:\SimoLIBR) are provided that contain the following information.

Criteria for Success

The following items must be completed to meet the minimum requirements for successfully completing the proof of concept.

What to do and How to do it

The objective of this project is to experience first-hand the issues that will be encountered and the solutions required when doing data conversions. Because most mainframe files contain a mixture of alpha-numeric fields (i.e. "USAGE IS DISPLAY" in COBOL terminology) and numeric fields that may be packed (COMP-3) or binary (COMP) the data conversion will need to be done at the field level.

For this project two directories will be used. The first directory (c:\SimoLIBR) contains the utility programs and copy files needed to generate the COBOL conversion programs. The c:\SimoLIBR directory will need to be included in the PATH environment variable. The second directory (c:\DataMig1) contains an Item Master file that is an EBCDIC encoded, Keyed-indexed file (c:\DataMig1\DataEbc1\ItemEbc1.DAT). A command file (c:\DataMig1\Converts\ItemE2Aexample.CMD) is provided for generating the two COBOL conversion programs. These two programs will be used to convert the EBCDIC encoded file into an ASCII encoded file. Two copy files (c:\DataMig1\CobCpy1\ItemCB01.CPY and c:\DataMig1\CobCpy1\CustCB01.CPY) are provided and define the data structure of the records in the Item Master File and the Customer Master File.

The following is a series of tasks that must be accomplished to successfully complete this learning session.

Task 01, Generate the COBOL Source Members for a Data File Conversion

Generate the COBOL source members that will do the actual Data File Conversion. From the c:\DataMig1\Converts directory run the command file ItemE2Aexample.CMD. This should generate the two COBOL source members that will be compiled and executed to do the data file conversion. The first generated program will do the file I/O to read the EBCDIC encoded file and write the ASCII encoded file. The second generated program will be called by the I/O program to do the actual conversion of each record from EBCDIC encoding to ASCII encoding. The integrity of mainframe numeric formatted fields will be maintained. The conversion of each record will be done at the field level based on the field definitions specified in a COBOL copy file.

Refer to the document for Data File Convert, Generate Source Code for additional help.

Note:  If this procedure does not run correctly the problem is usually with the c:\SimoLIBR directory. Either the SimoZAPS or SimoREC1 utility programs were not installed correctly or the c:\SimoLIBR directory is not in the PATH.

Task 02, Verify that Two COBOL Source Members were Created

Verify that two COBOL source members were created (ItKeKaC1.CBL and ItmE2AR1.CBL) and placed in the c:\DataMig1\COBOL directory.

Refer to the document for Data File Convert, View Source Code for additional help.

Note:  If task 1 is successful then the two COBOL source members should be automatically displayed with NotePAD. You may view the source code and then close the NotePAD windows. Do not change the source code.

Task 03, Create a Net Express Project

Create a Net Express project in the c:\DataMig1 directory.

Refer to the Start Net Express and Create a Project document for additional help.

Task 04, Add the Two COBOL Source Members to the Project

Add the two COBOL source members to the project.

For help in how to add source members to the project refer to the Add Source Member to Net Express Project document for additional help.

Task 05, Compile the Two COBOL Source Members

This task will describe and demonstrate how to compile the two COBOL programs.

For help in compiling the programs refer to the Compile for Execute with Animate section of this document.

If the compile is not successful refer to the Pointing to a Copy File Directory document for additional help.

Task 06, Animate the ITKEKAC1 program

Describe and demonstrate how to animate the ITKEKAC1 program.

For help with animation refer to the Animate the Conversion Program document for additional help.

When the OPEN for the input or output file fails refer to Mapping the COBOL File Name to a Fully Qualified PC File Name for additional help.

Refer to the File Status Codes for a description of a non-zero return code.

Task 07, Data File Editor

Use the Data File Editor to view the ASCII-encoded Item file (ItemAsc1.DAT) by first adding the file to the Net Express project.

For assistance with this process refer to Using the Data File Editor for additional help.

Task 08, Idiosyncrasies of Binary Data

Refer to The Idiosyncrasies of BINARY or COMP Data for additional information about why columns 82-83 did not translate correctly.

Task 09, Compiler Directives

Set the Compiler Directives to correct the failure in the conversion from EBCDIC-encoding to ASCII-encoding.

If the columns are incorrect refer to Setting Compiler Directives within a Project for additional help.

Task 10, View EBCDIC-encoded Item File and Record Structure

View the EBCDIC encoded Item file using the Data File Editor and Map Display to a .STR file (a .STR file is created from a Working Storage definition of the record layout, this is usually a copy file).

Refer to Data File Editor and Structured Layout for additional help.

Additional Information

This section provides additional information or insight into some of the tasks performed by this exercise. There are numerous things to consider when converting data files from an EBCDIC encoded format to an ASCII encoded format. Maintaining numeric integrity across all the different formats used on a mainframe (i.e. EBCDIC encoding with signed, zoned-decimal, COMP and COMP-3) requires special handling at the field level.

Numeric Fields, BINARY or COMP Format

This link provides additional information about Binary or COMP (i.e. COMPUTATIONAL) fields. The Binary or COMP Format, Description and Discussion document provides additional information for converting this type of field or data string.

Numeric Fields, Packed-Decimal or COMP-3 Format

This link provides additional information about Packed Decimal or COMP-3 (i.e. COMPUTATIONAL-3) fields. The Packed-Decimal or COMP-3 Format, Description and Discussion document provides additional information about this type of field or data string.

Numeric Fields, Zone Decimal Format

This link provides additional information about Zone Decimal fields. The unsigned, zone decimal fields are text fields and are assumed to be a simple, no risk conversion. However, it is best to treat unsigned fields as signed since it is possible for a signed zone value to end up in an unsigned field and as long as it is a positive value the mainframe will process the field.

Signed fields that are in a zone decimal format need special handling and should not be handled as text data. The Zoned Decimal Format, Description and Discussion document provides additional information for converting this type of field or data string.

Numeric Fields, Edited Format

Printing or displaying a numeric field can be a challenge because of all the different formats. This is usually done by using an edited numeric field. An Edited Numeric Field will make all the numeric values actual digits and provides for the insertion of commas and an explicit decimal point.

Summary

The purpose of this session is to describe the techniques and effort involved to convert a mainframe EBCDIC-encoded data file to a Windows and Micro Focus ASCII-encoded file while maintaining mainframe numeric integrity. This session is available as an on site class or an on line, self-study course.

Software Agreement and Disclaimer

Permission to use, copy, modify and distribute this software, documentation or training material for any purpose requires a fee to be paid to SimoTime Technologies. Once the fee is received by SimoTime the latest version of the software, documentation or training material will be delivered and a license will be granted for use within an enterprise, provided the SimoTime copyright notice appear on all copies of the software. The SimoTime name or Logo may not be used in any advertising or publicity pertaining to the use of the software without the written permission of SimoTime Technologies.

SimoTime Technologies makes no warranty or representations about the suitability of the software, documentation or learning material for any purpose. It is provided "AS IS" without any expressed or implied warranty, including the implied warranties of merchantability, fitness for a particular purpose and non-infringement. SimoTime Technologies shall not be liable for any direct, indirect, special or consequential damages resulting from the loss of use, data or projects, whether in an action of contract or tort, arising out of or in connection with the use or performance of this software, documentation or training material.

Downloads and Links

This section includes links to documents with additional information that are beyond the scope and purpose of this document. The first group of documents may be available from a local system or via an internet connection, the second group of documents will require an internet connection.

Note: A SimoTime License is required for the items to be made available on a local system or server.

Current Server or Internet Access

The following links may be to the current server or to the Internet.

Note: The latest versions of the SimoTime Documents and Program Suites are available on the Internet and may be accessed using the icon. If a user has a SimoTime Enterprise License the Documents and Program Suites may be available on a local server and accessed using the icon.

    Explore The SIMOPATH Series starting at the home page to view information about training sessions and learning materials available from SimoTime Enterprises.

    Explore the capabilities of the SimoZAPS Utility Program. This includes generating a COBOL program that will do the conversion of sequential and VSAM (KSDS) files between EBCDIC and ASCII while maintaining mainframe (or COBOL) numeric formats and integrity. SimoZAPS can also read a sequential file in EBCDIC format and create an ASCII/CRLF file or VSAM KSDS file in ASCII format. The conversion tables may be viewed or modified to meet unique requirements. The Hexcess/2 function provides the capability of viewing, finding or patching the contents of a file in hexadecimal.

    Explore How to Document Record Layouts or other Data Structures using a COBOL copy file. For reference or documentation purposes SimoREC1 creates an ASCII/Text file and an HTML document from a COBOL copy file. The utility provides the length of a field and the position of the start of the field within a record. Also, the actual physical length for group items, packed fields, binary fields and tables will be calculated.

    Explore The Binary or COMP format for numeric data strings. This numeric structure is supported by COBOL and may be explicitly defined with the "USAGE IS COMP" or "USAGE IS BINARY" clause.

    Explore The Edited for Display format for numeric data strings. This numeric structure is supported by COBOL and may be used with an edit-mask to prepare the presentation for readability by human beings.

    Explore The Packed-Decimal or COMP-3 format for numeric data strings. This numeric structure is supported by COBOL and may be explicitly defined with the "USAGE IS COMP-3" clause.

    Explore The Zoned-Decimal format for numeric data strings. This numeric structure is the default numeric for COBOL and may be explicitly defined with the "USAGE IS DISPLAY" clause.

    Explore commonly used formats and processing techniques for managing various numeric formats available on the mainframe.

    Explore the Numbers Connection for additional information about the structure and processing of numeric data items (or numeric fields).

    Explore An Enterprise System Model that describes and demonstrates how Applications that were running on a Mainframe System and non-relational data that was located on the Mainframe System were copied and deployed in a Microsoft Windows environment with Micro Focus Enterprise Server.

    Explore an Extended List of Software Technologies that are available for review and evaluation. The software technologies (or Z-Packs) provide individual programming examples, documentation and test data files in a single package. The Z-Packs are usually in zip format to reduce the amount of time to download.

    Explore The ASCII and EBCDIC Translation Tables. These tables are provided for individuals that need to better understand the bit structures and differences of the encoding formats.

    Explore The File Status Return Codes to interpret the results of accessing VSAM data sets and/or QSAM files.

    Explore the Hexadecimal Dump capabilites using COBOL to dump the content of a data string. This suite of programs includes a sample program that calls the SIMODUMP program to do the actual formatting for the hexadecimal dump information.

Internet Access Required

The following links will require an internet connect.

A good place to start is The SimoTime Home Page for access to white papers, program examples and product information. This link requires an Internet Connection

Explore The Micro Focus Web Site for more information about products and services available from Micro Focus. This link requires an Internet Connection.

Glossary of Terms

    Explore the Glossary of Terms for a list of terms and definitions used in this suite of documents and white papers.

Comments or Feedback

This document was created and is maintained by SimoTime Technologies. If you have any questions, suggestions, comments or feedback please use the following contact information.

We appreciate hearing from you.

Company Overview

SimoTime Technologies was founded in 1987 and is a privately owned company. We specialize in the creation and deployment of business applications using new or existing technologies and services. We have a team of individuals that understand the broad range of technologies being used in today's environments. Our customers include small businesses using Internet technologies to corporations using very large mainframe systems.

Quite often, to reach larger markets or provide a higher level of service to existing customers it requires the newer Internet technologies to work in a complementary manner with existing corporate mainframe systems. We specialize in preparing applications and the associated data that are currently residing on a single platform to be distributed across a variety of platforms.

Preparing the application programs will require the transfer of source members that will be compiled and deployed on the target platform. The data will need to be transferred between the systems and may need to be converted and validated at various stages within the process. SimoTime has the technology, services and experience to assist in the application and data management tasks involved with doing business in a multi-system environment.

Whether you want to use the Internet to expand into new market segments or as a delivery vehicle for existing business functions simply give us a call or check the web site at http://www.simotime.com