Testaid execution analyser for Cobol

TestAid

What is TestAid?

When a program executes, the only way to determine “what happened” is to evaluate the input and output. Did you ever want to know what happened inside the program when it was running? TestAid now provides you with the answers.

In addition to this it continuously monitors and provides real time information as testing progresses. This is useful in the following areas:

Project Management
Quality Assurance
Auditing
Development Testing

TestAid is useful for program development and all types of program/system testing. However, for Euro and other conversion testing it provides an extremely important audit trail, which provides proof of adequate testing for a system.

How does TestAid work?

The underlying aim of a typical program is to take decisions. After a decision a certain section of the program is executed based on the outcome of the decision. Therefore the flow of the program is constantly being changed by decisions within the program. It follows, that one can safely say that the more complex a program is, the more decisions it contains. In general, this is more accurate than saying the more lines that the program has, the more complex the programs is.

The amount of testing required to prove that a program is working correctly is more a function of how many decisions it contains, than how many lines of code it has. Also knowing that a certain line of code has executed is rather less meaningful than knowing that a section of code has been executed because a certain condition was or was not met. As decisions can have more than two outcomes it is important to track each section of code for each outcome.

It is much more useful to know what percentage of the decisions were executed within a program than what percentage of the program lines were executed.

TestAid breaks a program up into logical paragraphs (sections as described above), wherever it find that the flow of the program could change.

1.      DISPLAY "STARTED"
2.      OPEN F1
3.      READ F1 AT END
4.              DISPLAY "END"           <- A.
5.      NOT AT END
6.              IF F1-REC (1:1) = "1"   <- B.
7.                     DISPLAY "COL1=1" <- C.
8.              END-IF
9.      END-READ
10.    CLOSE F1

In this sample above there are three possible execution paths and therefore three logical paragraphs marked A. B. and C.

These logical paragraphs are identified for tracking purposes. When the program is recompiled and executed, a count is maintained for each logical paragraph and upon termination of the program statistics are written away to a file or database. These statistics are accumulated for each run of the program and it’s execution can be constantly monitored.

Program listings can be generated clearly showing the position of the logical paragraphs together with the number of times they have been executed. If a logical paragraph has not been executed, it is clearly marked for easy viewing. Alternatively one can request a listing showing the code that was not executed. In addition to this, if certain lines of code are tagged (as in conversions or EURO remediation), it is possible to obtain a listing of the unexecuted tagged code.

Execution coverage statistics can at any time be extracted, by program, by task/job, by user, by system or a combination of these. Statistics can also be exported to a PC for further analysis or graphical representation.

Performance and system overhead is an issue with any tool that attempts to monitor execution. Tools that attempt to monitor every line, besides missing the important things (explained above), are extremely costly as far as system overhead is concerned. Simple logic tells one that if one tries to monitor every line, a quantity of overhead is required for each and every line. The approach taken by TestAid requires the least system overhead, as logical paragraphs tend to be many lines of code and only one overhead is required for each paragraph. Should it be required it is feasible to run TestAid in production.

What are the benefits of using TestAid?

TestAid is an essential tool to answer the following questions

Project Managers – How far has the testing phase progressed?
Quality Assurance – How extensive is the testing effort?
Developers – Has all of the code for the program been tested?
Change Management – Has the required testing coverage been achieved to move this program or system into production?
Business Users – Were the test plans adequate to sign-off the system?

What do some of the reports look like?

Execution Statistics Report:

Program %Exec Count/Paras Compile_Date
-------- ----- ----------- ------------

TEST1 * 44% 0004/0009 199811031330

TEST2 * 20% 0005/0020 199812041022

TEST3 * 34% 0062/0180 199309021405

Execution Listing Report:

Program=TEST1                  Exec= 44% Paras=0004/0009 Compiled=199811031330

               DECLARATIVES.
               000-DECL-SEC SECTION.
               USE AFTER STANDARD ERROR PROCEDURE ON F1.
****> 0           DISPLAY "FILE I/O ERR".
               END DECLARATIVES.

               010-MAIN SECTION.
       1            OPEN INPUT F1
                   READ F1 AT END
****> 0                DISPLAY "FILE EMPTY"
                       EVALUATE W-ITEM
                       WHEN "A"
                       WHEN "B"
****> 0                    DISPLAY "A OR B"
                       WHEN OTHER
****> 0                    DISPLAY W-ITEM END-DISPLAY
                       END-EVALUATE
                   NOT AT END
       1                IF F1-REC (8:2) = "ID"
       1                    DISPLAY "COBOL SOURCE"
                       ELSE
****> 0                    DISPLAY "NOT COBOL SOURCE"
                       END-IF
                   END-READ
                   CLOSE F1.
               010-EXIT.
       1            STOP RUN.