Training Module 2: Performance Engine Overview

The following slides are used in this on-line video :

Slide 1

Welcome to the training course on IBM Scalable Architecture for Financial Reporting (or SAFR). This is module 2, “Performance Engine Overview.”

Slide 2

This module provides you with an introduction to the SAFR Performance Engine and how to use it. By the end of this training, you should be able to:

Identify the phases of the Performance Engine
Identify the main inputs and outputs for each phase
Give a high-level overview of the reports from each phase, and
Run a Performance Engine job stream

Slide 3

As we noted in the “Introduction to SAFR Views” module of this training course, SAFR consists of two software components: a PC-based Workbench and a mainframe-based batch process known as the Performance Engine. Developers use the Workbench to build applications that are stored in a metadata repository in an IBM® DB2® database. These applications are then executed by the Performance Engine, which reads data from source files or databases, transforms it, and writes it to output files. In this sense, SAFR is an application development tool and is not fundamentally different from any other tool or language.

Slide 4

The Performance Engine comprises six phases:

The SELECT phase, where the desired views are selected from the Metadata Repository
The COMPILE phase, where the selected views are converted to an executable form
The LOGIC phase, where logic from multiple views is consolidated and optimized for execution
The REFERENCE phase, where values are retrieved from lookup tables and optimized for execution
The EXTRACT phase, where data from source files is retrieved, merged with lookup data, and transformed for output, and
The optional FORMAT phase, where data is sorted, summarized, and formatted if necessary

Slide 5

The primary SAFR program in the Select phase is GVBMR86 (The names of all programs in the SAFR Performance Engine start with the letters “GVB,” so it is common to refer to this program as MR86). MR86 reads a list of view numbers and finds the corresponding views in the SAFR Metadata Repository. It then processes these views and converts them to an Extensible Markup Language (or XML) file. It also produces a control report known as the MR86 Report, which indicates successful completion or displays errors that may have to be corrected before proceeding.

Slide 6

The view numbers for the desired views are placed in the JCL, using a DD name called VIEWLIST. In this example, we are specifying view 90, which is the view we created in the “Introduction to SAFR Views” module of this training course.

Slide 7

For a view to be selected, it must first be set to Active status in the Workbench. If the view is not active, the MR86 Report will contain the error message “View is inactive.” A view that is inactive cannot be run.

Slide 8

When errors occur, MR86 also issues a condition code of 8 for the job step, which typically causes the remaining job steps to be skipped.

Slide 9

Once the view is activated in the SAFR Workbench, MR86 can be rerun and the view should be successfully selected for processing. If no other errors are encountered, the MR86 Report does not contain the word “ERROR” at the beginning of any row, and MR86 returns a condition code of 0 for the job step, allowing remaining jobs steps to run.

Slide 10

The primary SAFR program in the Compile phase is GVBMR84. MR84 reads the XML file created by the previous step and converts it to a form that can be executed by the Extract phase later. This new file is called the View Definition Parameters (or VDP) file. Because multiple versions of the VDP file are used in the SAFR job stream, this one is referred to as the MR84 VDP to indicate the program that wrote it. MR84 also produces a control report known as the MR84 Report.

Slide 11

The MR84 Report indicates successful completion or displays XML syntax errors and view compile errors that have to be corrected before proceeding. If there are no errors, the process continues.

The MR84 Report also shows how columns in the view are translated into a specialized SAFR construct known as a logic table.

Slide 12

The primary SAFR program in the Logic phase is GVBMR90. MR90 reads the MR84 VDP file created by the previous step and creates a new VDP and logic tables that are used by the next steps, the Reference and Extract phases. The new VDP is called the MR90 VDP to distinguish it from the input VDP file. The logic tables are known as the MR90 JLT (for “Join Logic Table”) and the MR90 XLT (for “the Extract Logic Table”).

The next programs in the Logic phase, which are not shown here, read these files and produce converted versions known as the MR77 VDP, the MR76 JLT, and the MR76 XLT.

Slide 13

The MR90 Report displays summary statistics for the Logic Table Creation process. In this case, the Extract Logic Table contains nine records. This view included no joins, so no records were written to the Join Logic Table. The report also displays, in detail format, the contents of the Extract Logic Table and the Join Logic Table if applicable.

Slide 14

The primary SAFR program in the Reference phase is GVBMR95. MR95 reads the MR77 VDP file and the MR76 JLT file created by earlier steps and one or more Reference Data files. It then produces a Reference Extract Header (or REH) that describes the format of the reference data being processed. It also creates one Reference Extract Detail (or RED) file for each Reference Data file read. These files contain only the data required to perform joins in the Extract phase. Finally, MR95 produces a control report known as the MR95 Report, which indicates successful completion or displays errors that may have to be corrected before proceeding.

Slide 15

If this job stream does not access any Reference Data files, the MR95 Report displays a warning message saying that there is an “empty or abbreviated logic table.” If you do not intend to access any Reference Data files, this is considered a normal condition and you may proceed to the next step.

To be processed correctly, reference data must be sorted in key sequence and have no records with duplicate keys.

Slide 16

The primary SAFR program in the Extract phase is GVBMR95, which is the same program that is run in the Reference phase. In reviewing the job stream output, you should take care not to confuse the outputs of the two executions of MR95.

MR95 reads the MR77 VDP file, the MR76 XLT file, and the REH, and RED files created by earlier steps, along with source data for the process (Source data is sometimes referred to as business event data or just event data).

MR95 executes various transformations and, depending on specifications in the selected views, produces one or more View Output files. It may also produce one or more Extract Work files, which are temporary files processed by the Format phase. Finally, it produces an MR95 Report, which presents summary statistics for the process.

Slide 17

Before running the Extract phase job, you must make sure that DD statements for any required input files are included, containing the data to be scanned and extracted according to the view requirements. The DD names (such as CUSTOMER or ORDER001) must match those in the Workbench physical files referenced by the views being run.

Slide 18

Similarly, you also must ensure that DD statements for any required output files are included. These DD names are determined by view parameters in the Workbench.

Slide 19

The Extract phase MR95 Report displays statistics that are useful for audit trails and performance tuning. In this example, we can see that 12 records were read from the ORDER001 file, 12 records were processed by view 90, and 12 records were written to the OUTPUT01 file.

Slide 20

A secondary program in the Extract phase is GVBUT90, which produces the UT90 Report. This report displays information from the Extract Logic Table (XLT), which shows each step that is executed in a data transformation. This is useful information for debugging views. A more detailed discussion of the logic table is presented in the training module entitled “Basic Debugging.”

Slide 21

The primary SAFR program in the Extract phase is GVBMR88. MR88 reads the MR77 VDP, REH, and RED files created by earlier steps, along with the temporary Extract Work files for the process. It then sorts, summarizes, and formats the data, producing one or more View Output files. Additional details will be provided in the training module entitled “Introduction to the Format Phase.”

Slide 22