Test Generator for ISO Modula-2
By Tom Breeden (tmb@virginia.edu)
May 20, 2004
Overview
tgM2 is a program to aide in unit testing of Modula-2 software. It takes as input
a user-prepared test script (*.ts file) which specifies one or more tests of a procedure or
code block to be executed and reported upon. tgM2 writes out a Modula-2 program for this
test suite. The output program itself writes its results to standard output.
tgM2 owes its existence and most of its architecture to the Ada 95 test driver generator
programtg for Ada written by Andre Spiegel, along with the version
of tg for Modula-2 (still an Ada 95 program) adapted to generate Modula-2 scripts done by Ralf Reissing.
I produced tgM2 for two reasons. First, I wanted to remove the necessity of obtaining an Ada
compiler in order to build the program. Second, I wanted to use ISO Modula-2 to restore the
capabilities of the original Ada driver generation that had been lost in adaptation to a pre-ISO
M2 language base. Specifically, these depended on the EXCEPT language construct in the ISO definition.
Exception handing provides the possibility of generating more robust test scripts as well as the ability
to test that situations which should cause exception execution actually do so.
The program was developed and tested on Win2K using Stony Brook Modula-2 and on an Amiga using a semi-ISO (unreleased)
M2 compiler for Amiga (M68040).
The files in the download tgM2.zip
include the complete sources of tgM2 and a Windows executable, plus some demo script files, and support module sources.
My own use of tgM2 has just begun, so it must be considered to be in a beta state.
Your bug reports and other comments are appreciated.
Test Scripts
The test script you prepare is a text file (with extension ".ts") containing all the information
necessary for tgM2 to generate a "driver" program source file that can be compiled and linked
to run the test suite.
The different sections of the test script are:
- General Section - can occur once, at the beginning of the script.
- Fail/Error Handling Specification
- "stop" at first failure/error or "continue"
- Context
- the IMPORT statements necessary
- Global Definition
- variables used by all the test cases
- CONST and TYPE statements for global use.
- Procedures called from the Test Code section.
- Code Sections - can occur multiple times, and interspersed with Test Case Parts
- Code Section
- code to be executed before/between Test Cases.
- Code Sections and Test Case Sections
are executed in the order in which they appear in the script file.
- Test Case Sections - can occur multiple times to create a test suite
- Title
- descriptive title for the test case
- Definitions
- variables used by this test case
- CONST and TYPE statements for this test case.
- Procedures called within this test case only.
- Preparation Code
- statements executed prior to the test itself
- Test Code
- statements that perform the test itself
- Test Result Predicate(s)
- boolean expressions to be evaluated after the Test Code
- or within an EXCEPT block caused by the test code
- all must evaluate TRUE for the test to be marked "Pass"
- CleanUp Code
- statements executed after to the test itself
Test Script Layout
The *.ts file is a "column 1 significant" text file; ie, if a non-blank character
appears in column 1, it must be a "keyword" (such as "define", "test", "pass").
Other material is (mainly) Modula-2 statements or phrases, which can encompass more than one
line, so long as column 1 is blank.
Comments in the *.ts file are in M2 comment format, but are never passed onto the driver program.
(currently nested comments are not supported).
Most of the sections or subsections above are optional. However, there must
be a Test Case Title, a Test Code subsection, and a Test Result Predicate. Subsections must
appear in the order given above.
Keywords are not case sensitive
- General Section
- Keyword: fail_handling stop | fail_handling continue
Keyword: error_handling stop | error_handling continue
- Keyword: context <IMPORT statement> {<IMPORT statement>}
- Keyword: define <variables list>:<Type>; {<M2 definitions and procedures >}
- Note: No VAR in front of the initial variables list.
- Code Sections
- Keyword: code <M2 definitions and code>
- Note: Will be wrapped into a procedure constructed by tgM2
- Test Case Sections
- Keyword: ***** <text>
- Note: The test case will be wrapped into a procedure constructed by tgM2
- Keyword: define <variables list>:<Type>; {<M2 definitions and procedures >}
- Note: No VAR in front of the initial variables list.
- Keyword: prepare <M2 code>
- Keyword: test <M2 code>
- Keyword: pass <boolean expression>
Keyword: pass exception <boolean expression>
- Note: the "pass exception" predicate is placed into the procedure's EXCEPT block.
- Note: Multiple instances of either of the pass keywords may appear.
- Keyword: cleanup <M2 code>
(* Test script for some Arithmetic expressions with a Modula-2 compiler *)
(* 051704 for Stony Brook *)
fail_handling continue
error_handling stop
context FROM M2EXCEPTION IMPORT IsM2Exception;
<*/NOOPTIMIZE*> (* Otherwise, some of the simple tests below disappear! *)
***** INTEGER Overflow
define Result, I, J, K :INTEGER16;
prepare <*/PUSH*>
<*/CHECK:O*>
Result := 0;
I := 30000; J := -30000;
test Result := I-J;
pass exception IsM2Exception()
cleanup <*/POP*>
***** INTEGER Overflow without checking
define Result, I, J, K :INTEGER16;
prepare <*/PUSH*>
<*/NOCHECK:O*>
Result := 0;
I := 30000; J := -30000;
test Result := I-J;
pass TRUE
cleanup <*/POP*>
|
tgM2: A Test Driver Generator for Modula-2 programs (v0.9-051804).
Usage: tgM2 [options...] script_file [driver_file]
Options template is "-f=-fail/K,-p=-pass/K,-script/A,-driver,-overwrite/S,-h=-help/S"
/A = always required
/K = option keyword required
/S = boolean switch
-f Specifies how failed test cases will be reported
by the testdriver. may be one of
off No output.
numbers Only test case numbers, followed by the string
"FAIL.".
titles Numbers and titles, followed by "...FAIL."
on the next line.
full Numbers and titles, "...FAIL" and a short
explanation on the next line.
This is the default.
-p Same as "-f", but for passed test cases.
Default is "numbers".
-overwrite Overwrite any existing driver file of the same name
The script_file must have the extension ".ts".
If the driver_file is not specified, the name is constructed from the
script_file with extension changed to ".mod".
|
Test Scripts
Demo1.ts from Ada 95 tg package.
ArithExpressions.ts some arithmetic overflow tests with exceptions.
TsttgM2Parser.ts a more complex script doing a bit of testing of tgM2's parsing modules.
Program Modules
tgM2
tgM2TranslatorISO
tgM2Parser
tgM2Scanner
tgM2Exceptions
tgM2Utils
tgM2Version
tgM2.exe Win32 executable
tgM2.sbp Stony Brook Modula2 project file
ISO Lib Support Modules
ArgsSupport specification, reading, and initial parsing of command line arguments.
TextIOHelper eases the use of the ISO standard TextIO input routines.
Null delimited static strings
Str0 Basic procedures.
Str1 More procedures.
Dynamic strings
DynStr0 Basic procedures, size fixed at create time.
DynStr1 Common procedures.
DynStr1b some Strings module flavor procs.
DynStr1NC Compare procedures, case insensitive.
DynStr1C Common procedures, one character parameter.
DynStr3 As DynStr0 and DynStr1, but size will be reallocated on assignment.
TextBlocks In-memory, line-oriented, dynamically-extending blocks of text.
List processing
ListV2Processor Simple generalized list processing module.
Misc
MaxMin Maximum and Minimum functions for CARDs, INTs, REALs.
Debugging0 Assert and Debug procs.
TBChans used only by tsttgM2Parser.ts
Notes
- "fail" vs. "error" - A test case is marked "fail" if any of the pass or pass exception predicates are false.
It is also marked "fail" if an exception occurred within the test subsection but there was no pass exception keyword,
or, similarly, if there was a pass exception keyword but no exception occurred within the test subsection.
If there is an exception anywhere outside of the test subsection then the test case (and the overall run result) is marked
"error".
- The Windows executable was created in the Stony Brook M2 v4, build 29 system.
- Using an optimizing compiler, for some very simple test and pass combinations it
may be necessary to turn off optimization or take other steps to insure that the compiler does not
remove some of your statements. (see ArithExpressions.ts)
- The supplied module Debugging0's implementation is Windows specific.
|