Testaid execution analyser for Natural/Adabas

TestAid

What is TestAid?

When a program executes, the only way to determine “what happened” is to evaluate the input and output. Did you ever want to know what happened inside the program when it was running? The TestAid now provides you with the answers.

In addition to this it continuously monitors and provides real time information as testing progresses. This is useful in the following areas:

Project Management
Quality Assurance
Auditing
Development Testing

TestAid is useful for program development and all types of program/system testing. However, for Euro and other conversion testing it provides an extremely important audit trail, which provides proof of adequate testing for a system.

How does TestAid work?

The underlying aim of a typical program is to take decisions. After a decision a certain section of the program is executed based on the outcome of the decision. Therefore the flow of the program is constantly being changed by decisions within the program. It follows, that one can safely say that the more complex a program is, the more decisions it contains. In general, this is more accurate than saying the more lines that the program has, the more complex the programs is.

The amount of testing required to prove that a program is working correctly is more a function of how many decisions it contains, than how many lines of code it has. Also knowing that a certain line of code has executed is rather less meaningful than knowing that a section of code has been executed because a certain condition was or was not met. As decisions can have more than two outcomes it is important to track each section of code for each outcome.

It is much more useful to know what percentage of the decisions were executed within a program than what percentage of the program lines were executed.

TestAid breaks a program up into logical paragraphs (sections as described above), wherever it find that the flow of the program could change.

DEFINE DATA LOCAL
1 EMPLOYEES VIEW OF EMPLOYEE
  2 NAME
  2 SEX
*
1 #MALE   (N5)
1 #FEMALE (N5) LOCAL USING PEXPDA
END-DEFINE
FIND EMPLOYEES WITH NAME = 'SMITH'                        <=== 1
IF NO RECORDS FOUND                                       <=== 2
    DISPLAY 'NO RECORDS FOUND'
    STOP                                                  <=== 3
  END-NOREC
*
IF SEX = 'M'                                              <=== 4
    ADD 1 TO #MALE
  ELSE                                                    <=== 5
    ADD 1 TO #FEMALE
  END-IF
*
END-FIND
*
WRITE 'TOTAL MALES   :' #MALE                             <=== 6 
WRITE 'TOTAL FEMALES :' #FEMALE
END

In this sample above there are six possible execution paths and therefore six logical paragraphs marked 1-6.

These logical paragraphs are identified for tracking purposes. When the program is recompiled and executed, a count is maintained for each logical paragraph and upon termination of the program statistics are written away to a file or database. These statistics are accumulated for each run of the program and it’s execution can be constantly monitored.

Program listings can be generated clearly showing the position of the logical paragraphs together with the number of times they have been executed. If a logical paragraph has not been executed, it is clearly marked for easy viewing. Alternatively one can request a listing showing the code that was not executed. In addition to this, if certain lines of code are tagged (as in conversions or EURO remediation), it is possible to obtain a listing of the unexecuted tagged code.

Execution coverage statistics can at any time be extracted, by program, by task/job, by user, by system or a combination of these. Statistics can also be exported to a PC for further analysis or graphical representation.

Performance and system overhead is an issue with any tool that attempts to monitor execution. Tools that attempt to monitor every line, besides missing the important things (explained above), are extremely costly as far as system overhead is concerned. Simple logic tells one that if one tries to monitor every line, a quantity of overhead is required for each and every line. The approach taken by TestAid requires the least system overhead, as logical paragraphs tend to be many lines of code and only one tracking overhead is required for each paragraph. Should it be required it is feasible to run the TestAid in production.

What are the benefits of using TestAid?

TestAid is an essential tool to answer the following questions

Project Managers – How far has the testing phase progressed?
Quality Assurance – How extensive is the testing effort?
Developers – Has all of the code for the program been tested?
Change Management – Has the required testing coverage been achieved to move
this program or system into production?
Business Users – Were the test plans adequate to sign-off the system?

What do some of the reports look like?

Execution Statistics Report:

Program %Exec Count/Paras Compile_Date
-------- ----- ----------- ------------

TEST1 44% 0004/0009 19981103133003

TEST2 20% 0005/0020 19981204102246

TEST3 34% 0062/0180 19930902140529

Execution Listing Report:

PROGRAM    : PEXEMPLY
USER       : EUGENE
TIME STAMP : 19990303175047
EXECUTION  = 66%
PARAGRAPHS = 0004 / 0006
-------------------------------------------------------

EXEC COUNT                        SOURCE CODE
----------                        -----------
        1     FIND EMPLOYEES WITH NAME = 'SMITH'
              IF NO RECORDS FOUND
NOT EXECUTED  DISPLAY 'NO RECORDS FOUND'
NOT EXECUTED    STOP
              END-NOREC
              IF SEX = 'M'
       13        ADD 1 TO #MALE
               ELSE
        6        ADD 1 TO #FEMALE
              END-IF
              END-FIND
              WRITE 'TOTAL MALES   :' #MALE
              WRITE 'TOTAL FEMALES :' #FEMALE
        1     END