APPENDIX D: DIAGNOSTIC ASSESSOR README FILE

INTRODUCTION:

The Diagnostic Assessor tests contain examples of illegal or questionable programs, so that the compilation system's diagnostic messages may be generated for analysis. The suite of tests is designed to provide compiler warnings, compiler errors, link-time messages, and run-time messages. Since the LRM does not specify the form or content of diagnostic messages, the following questions are provided in the summary report form ("yd_tmplt.txt") as guidelines for analysis:

* Is a diagnostic message printed?

* Is the message in the general area of difficulty?

* Is the message at the correct specific location?

* Does the text of the message clearly define the difficulty?

* Is relevant non-local information listed where appropriate?

* Is error recovery appropriate?

The Diagnostic Assessor tests are run by command files. Note that the extension is ".com" for VAX and ".unx" for UNIX. The user should assume that instructions regarding files with the extension ".com" also refer to the equivalent ".unx" files.

PREPARE TO TEST:

* Carry out these preliminary steps.

+ Print the summary report form ("yd_tmplt.txt") so that it may be manually filled out, or bring it up in a window to be completed with a text editor. You may prefer to copy this file to a filename of your choice.

+ For VMS systems define the directory of your choice to be "aces_working_directory". You may want to do this in the "login.com" file.

* Have these files available:

+ All "yd_*.ada" files (rename with appropriate file suffix for implementation being tested)

+ The command files:

- "zc_delsb.com"

- "yd_drivr.com"

- "yd_linkr.com"

- "yd_fillr.com"

- "yd_compl.com"

+ The global files (only if Pretest Step 5 has not been run). It is not necessary to recompile these files if Pretest Step 5 is already completed.

- "zg_glob1.ada"

- "zg_glob4.ada"

* Modify the "yd_drivr.com" file:

+ Modify the command to compile yd_rt05b to use the system's nooptimize option.

+ yd_lt04a is designed to test the system's response to disk space problems. Some systems allow the user to set a disk quota and thus artificially exhaust the disk space. If the system under test has a disk quota command, set it to the size of the current value (in "yd_drivr.com", before the command "@yd_linkr yd_lt04a"; in "yd_drivr.unx", before the command "yd_compl.unx yd_lt04a"). Note that, in "yd_drivr.unx", all the commands for yd_lt04a are commented. If you wish to run this test (using either scenario), remove the appropriate comment delimiters. WARNING: On some UNIX systems, using all available disk space can cause a system crash.

If the system does not have a disk quota command, the disk space can be filled by altering the "yd_driver.com" file to call "yd_fillr.com", then executing the yd_linkr command, and then deleting the temporary file created. Here is the sequence for VMS:

$ @yd_fillr

$ @yd_linkr yd_lt04a

$ delete tmp*.dsk;*

* Modify the three lower-level files if necessary: "yd_fillr.com", "yd_linkr.com", and "yd_compl.com".

EXECUTE THE TESTS:

* Submit a batch job to run the "yd_drivr.com" file to execute compiler message and warning tests, and link-time error tests. The command file will also compile and link the run-time tests in preparation for their execution by the user. System messages will be written to the file "yd_dtest.lis", which is specified in the command file.

The test files listed below (compiler error, compiler warning, and link-time error) are executed by the command file and do not require intervention by the user:

+ yd_ce01a: Nongraphic character inserted in the source code.

+ yd_ce02a: Mismatched bracketing structure: an unmatched IF statement.

+ yd_ce02b: Mismatched bracketing structure: an unmatched END statement.

+ yd_ce03a: Naming conflicts: same type signature occurs more than once.

+ yd_ce03b: Naming conflicts: same object name used more than once.

+ yd_ce03c: Naming conflicts: duplication after error (the name of an erroneous declaration reused correctly later).

+ yd_ce03d: Naming conflicts: Same name referenced by USE.

+ yd_ce03e: Naming conflicts: incomplete type declaration.

+ yd_ce03f: Naming conflicts: inconsistent designators.

+ yd_ce03g: Naming conflicts: ambiguous references during instantiation.

+ yd_ce03h: Naming conflicts: ambiguous references in function calls.

+ yd_ce03i: Naming conflicts: ambiguous references in system calls.

+ yd_ce04(a-c): Circular order dependency.

+ yd_ce05(a,b): Self WITH: library contains version.

+ yd_ce05c: Self WITH: library does not contain version.

+ yd_ce06a: Invalid aggregate associations: missing and duplicate elements.

+ yd_ce07a: Invalid CASE alternatives: multiple mentions of alternatives.

+ yd_ce07b: Invalid CASE alternatives: alternatives not mentioned.

+ yd_ce08a: Uncalled entry: scope limited to one unit.

+ yd_ce08b: Uncalled entry: entry names exported from package.

+ yd_ce09a: Invalid assignment to a CONSTANT.

+ yd_ce10a: Visibility error within a subunit.

+ yd_ce11a: FOR i IN-1..10 LOOP


WARNING: This construct is LEGAL in Ada 95 and should NOT produce a diagnostic message on an Ada 95 implementation.


+ yd_ce12a: Cascading error messages: missing semicolon.

+ yd_ce12b: Cascading error messages: arithmetic lexical errors.

+ yd_ce12c: Cascading error messages: other arithmetic errors.

+ yd_ce12d: Cascading error messages: declarative errors.

+ yd_ce12e: Cascading error messages: misspelled name.

+ yd_ce13a: Suppress duplicate error messages.

+ yd_ce14a: Improper type qualification.

+ yd_ce15a: Improper type conversion.

+ yd_ce16a: Diagnostic limits: large non-Ada program.

+ yd_ce16b: Diagnostic limits: non-visible names.

+ yd_ce17(a,b): Inconsistent package body.

+ yd_cw01a: Exception propagation: local exception with no handler.

+ yd_cw01b: Exception propagation past visibility.

+ yd_cw02a: Reference to uninitialized variables in declarative region.

+ yd_cw02b: Reference to uninitialized variables after external procedure call.

+ yd_cw02c: Reference to uninitialized variables: referenced before defined.

+ yd_cw03a: Non-returning functions.

+ yd_cw04a: Peculiar parameter usage: undefined OUT mode formal parameter.

+ yd_cw04b: Peculiar parameter usage: possible undefined OUT parameter.

+ yd_cw04c: Peculiar parameter usage: unreferenced IN mode parameter.

+ yd_cw04d: Peculiar parameter usage: IN OUT mode only read from.

+ yd_cw05a: Compile time warning for violations: reference to out of range subscript.

+ yd_cw05b: Compile time warning for violations: all alternatives are closed.

+ yd_cw05c: Compile time warning for violations: non-pre-elaborated function.

+ yd_cw06a: Selective wait to raise PROGRAM_ERROR.

+ yd_cw07a: Unrecognized pragma.

+ yd_cw08a: Ignored pragma INLINE.

+ yd_cw09a: Improper pragma location.

+ yd_cw10a: Duplicate ACCEPT entries in a SELECT.

+ yd_cw11a: Superfluous delays in SELECT.

+ yd_cw12a: Excessively large data structure.

+ yd_cw13a: Unreferenced variables: local variables unreferenced.

+ yd_cw13b: Unreferenced variables: unreferenced exported names.

+ yd_cw14a: Dead variable assignments: redefined before used.

+ yd_cw14b: Dead variable assignments: successive assignments using variable as a temporary.

+ yd_cw15a: Unreachable code: statements following unconditional transfer statement.

+ yd_cw15b: Unreachable code: statements following conditional all of whose alternatives transfer control.

+ yd_cw15c: Unreachable code: impossible CASE alternatives.

+ yd_cw16a: Unneeded WITH clause.

+ yd_cw17a: Invalid dependency on parameter passing mode (copy/reference).

+ yd_cw18a: Notification of obsolescence.

+ yd_lt01a: Missing units in library.

+ yd_lt02(a,b): Improper subunit: misspelled parent name.

+ yd_lt02(c,d): Parent unit does not specify subunit.

+ yd_lt03(a-c): Link with obsolete units.

+ yd_lt04a: Exhaust library disk quota.

+ yd_lt05a: Omission of unreferenced subprograms: subprograms never referenced.

+ yd_lt05b: Omission of unreferenced subprograms: isolated, inter-referenced subprograms.

* At this point the command file, "yd_drivr.com", has compiled and linked all the files, including the run-time tests listed below. Those tests which are designed to prompt compile/link diagnostics were completed when the command file completed. However, the run-time tests will now need to be run interactively, one at a time, and the results analyzed.

Run the following tests manually. It is helpful to capture the output and system messages in a file.

+ yd_rt01a: Deadlock detection.

Be prepared to interrupt and terminate this program, which may run indefinitely.

+ yd_rt02a: Access errors: dereference through deallocated types.

+ yd_rt02b: Access errors: different type object access.

+ yd_rt02c: Access errors: access through bad address.

+ yd_rt02d: Access errors: access through deallocated object.

+ yd_rt03a: Exhausting storage.

May crash the system; be prepared to terminate it.

+ yd_rt04a: Unhandled exception from main procedure: no exception handler.

+ yd_rt04b: Unhandled exception from main procedure: unnamed RAISE statement.

+ yd_rt05a: Uninitialized variables: optimize.

+ yd_rt05b: Uninitialized variables: no optimize.

ANALYZE/RECORD THE RESULTS:

Each test problem ".ada" file has comments, describing the expected result of the test activity for the user's guidance in analysis. Compare the actual response with the expected response, and fill out the blanks in your copy of the summary report ("yd_tmplt.txt").

Inspect the system responses as follows:

* Compile, warning, and link-time tests - Compare the comments in the individual Ada files to the diagnostic messages written to "yd_dtest.lis".

* Problems yd_lt05a and yd_lt05b - Examine the corresponding map files. (See the comments in the ".ada" files for these tests.)

* Run-time tests - Examine the messages displayed on the screen or written to a file.

ADA COMPILER EVALUATION SYSTEM Version 2.0

DIAGNOSTIC ASSESSOR REPORT

Evaluator's Name:____________________________________Date:____________

Compiler:____________________________________________Version:_________

Operating

System:______________________________________________Version:_________

Host

Hardware:____________________________________________Model:___________

Disk

Devices:_____________________________________________Memory:__________

Math

Package:______________________________________________________________

Other

Information:__________________________________________________________

______________________________________________________________________

Question Results Expressed as Percentages

Compiler Compiler Link Run

Question Error Warning Time Time

_______________________________________________

1. ____ ____ ____ ____

2. ____ ____ ____ ____

3. ____ ____ ____ ____

4. ____ ____ ____ ____

5. ____ ____ ____ ____

6. ____ ____ ____ ____

DIAGNOSTIC QUESTIONS

1. Is a diagnostic message printed?

2. Is the message in the general area of difficulty?

3. Is the message at the correct specific location?

4. Does the text of the message clearly define the difficulty?

5. Is relevant non-local information listed where appropriate?

6. Is error recovery appropriate?

Key: y = yes, n = no, p = partial, x = not applicable.

Test Name Prefix Convention: yd_ce___ : Compiler Error

yd_cw___ : Compiler Warning

yd_lt___ : Link Time

yd_rt___ : Run Time

ACES Version 2.0

DIAGNOSTIC ASSESSOR

COMPILER ERROR

TEST NAME DIAGNOSTIC QUESTIONS

====================================================================

1 2 3 4 5 6

====================================================================

yd_ce01a: Nongraphic character _ _ _ _ x x

yd_ce02 : Mismatching bracketing structures

a: Unmatched IF statement _ _ _ _ _ x

b: Unmatched END _ _ _ _ _ _

yd_ce03 : Naming conflicts

a: Same type signature _ _ _ _ _ x

b: Same object name _ _ _ _ _ x

c: Duplication after error _ _ _ _ _ _

d: Same name referenced by USE _ _ _ _ _ x

e: Incomplete type declaration _ _ _ _ _ x

f: Inconsistent designators-body/spec _ _ _ _ _ x

g: Ambiguous reference- instantiation _ _ _ _ _ x

h: Ambiguous reference- function _ _ _ _ _ x

i: Ambiguous reference- system call _ _ _ _ _ x

yd_ce04(a-c): Circular order dependency _ _ _ _ _ x

yd_ce05 : Self with

a,b: Library contains version _ _ _ _ x x

c: Library does not contain version _ _ _ _ x x

yd_ce06a: Invalid named aggregate associations _ _ _ _ _ x

yd_ce07 : Invalid CASE alternatives

a: Multiple mentioned alternatives _ _ _ _ _ x

b: alternatives not mentioned _ _ _ _ _ x

yd_ce08a: Non-returning functions _ _ _ _ x x

yd_ce09a: Invalid assignment to a CONSTANT _ _ _ _ _ x

yd_ce10a: Visibility error within a subunit _ _ _ _ x x

yd_ce11a: FOR i IN -1..10 LOOP _ _ _ _ x x

ACES Version 2.0

DIAGNOSTIC ASSESSOR

COMPILER ERROR (cont.)

TEST NAME DIAGNOSTIC QUESTIONS

====================================================================

1 2 3 4 5 6

====================================================================

yd_ce12 : Cascading error messages

a: Missing semicolon _ _ _ _ x _

b: Arithmetic lexical errors _ _ _ _ x _

c: Other arithmetic errors _ _ _ _ x _

d: Declarative error _ _ _ _ _ _

e: Misspelled name _ _ _ _ _ _

yd_ce13a: Suppress duplicate error messages _ _ _ _ x _

yd_ce14a: Improper type qualification _ _ _ _ x x

yd_ce15a: Improper type conversion _ _ _ _ _ x

yd_ce16 : Diagnostic limits

a: Large non-Ada program _ _ _ _ x _

b: Non-visible names _ _ _ _ x _

yd_ce17(a,b): Inconsistent package body _ _ _ _ _ x

====================================================================

TOTAL "YES" and "PARTIAL" ANSWERS

FOR COMPILER ERROR DIAGNOSTICS _ _ _ _ _ _

DIVIDE

TOTAL BY PERCENTAGES

1 2 3 4 5 6

1. Is a diagnostic message printed? __ / 33 = _

2. Is the message in the general area

of difficulty? __ / 33 = _

3. Is the message at the correct

specific location? __ / 33 = _

4. Does the text of the message clearly

define the difficulty? __ / 33 = _

5. Is relevant non-local information

listed where appropriate? __ / 20 = _

6. Is error recovery appropriate? __ / 10 = _

ACES Version 2.0

DIAGNOSTIC ASSESSOR

COMPILER WARNINGS

TEST NAME DIAGNOSTIC QUESTIONS

====================================================================

1 2 3 4 5 6

====================================================================

yd_cw01 : Exception propagation past visibility

a: Local exception with no handler _ _ _ _ x x

b: Propagation past visibility _ _ _ _ x x

yd_cw02 : Reference to uninitialized variables

a: In declarative region _ _ _ _ x x

b: After external procedure call _ _ _ _ x x

c: Referenced before defined _ _ _ _ x x

yd_cw03a: Uncalled entry

a: Scope limited to one unit _ _ _ _ _ x

b: Entry names exported from package _ _ _ _ _ x

yd_cw04 : Peculiar parameter usage

a: Undefined OUT mode formal parameter _ _ _ _ x x

b: Possible undefined OUT parameter _ _ _ _ x x

c: Unreferenced IN mode parameter _ _ _ _ x x

d: IN OUT mode- only read from _ _ _ _ x x

yd_cw05 : Compile time warning for violations

a: Reference to out of range subscript _ _ _ _ _ x

b: All alternatives are closed _ _ _ _ x _

c: Non-pre-elaborated function _ _ _ _ _ x

yd_cw06a: Selective wait to raise PROGRAM_ERROR _ _ _ _ x x

yd_cw07a: Unrecognized pragma _ _ _ _ x x

yd_cw08a: Ignored PRAGMA INLINE _ _ _ _ _ x

yd_cw09a: Improper PRAGMA LOCATION _ _ _ _ x x

yd_cw10a: Duplicate ACCEPT entries in a SELECT _ _ _ _ x x

yd_cw11a: Superfluous delays in SELECT _ _ _ _ _ x

yd_cw12a: Excessively large data structure _ _ _ _ x x

yd_cw13 : Unreferenced variables

a: Strictly local variables unreferenced _ _ _ _ x x

b: Unreferenced exported names _ _ _ _ x x

ACES Version 2.0

DIAGNOSTIC ASSESSOR

COMPILER WARNINGS (cont.)

TEST NAME DIAGNOSTIC QUESTIONS

====================================================================

1 2 3 4 5 6

====================================================================

yd_cw14 : Dead variable assignments

a: Redefined before used _ _ _ _ x x

b: Successive assignments using _ _ _ _ x x

variable as a temporary

(No warning = Y)

yd_cw15 : Unreachable code

a: Statements following _ _ _ _ x x

unconditional transfer statement

b: Statements following _ _ _ _ x x

conditional all of whose alternatives

transfer control

c: Impossible CASE alternatives _ _ _ _ x x

yd_cw16a: Unneeded WITH clause _ _ _ _ x x

yd_cw17a: Invalid dependency on parameter

passing mode (copy/reference) _ _ _ _ x x

yd_cw18a,b:Notification of obsolescence _ _ _ _ _ x

ACES Version 2.0

DIAGNOSTIC ASSESSOR

COMPILER WARNINGS (cont.)

TOTAL "YES" and "PARTIAL" ANSWERS

FOR COMPILER ERROR DIAGNOSTICS _ _ _ _ _ _

DIVIDE

TOTAL BY PERCENTAGES

1 2 3 4 5 6

1. Is a diagnostic message printed? __ / 31 = _

2. Is the message in the general area

of difficulty? __ / 31 = _

3. Is the message at the correct

specific location? __ / 31 = _

4. Does the text of the message clearly

define the difficulty? __ / 31 = _

5. Is relevant non-local information

listed where appropriate? __ / 6 = _

6. Is error recovery appropriate? __ / 1 = _

ACES Version 2.0

DIAGNOSTIC ASSESSOR

LINK TIME ERRORS

TEST NAME DIAGNOSTIC QUESTIONS

====================================================================

1 2 3 4 5 6

====================================================================

yd_lt01a: Missing units in library _ _ _ _ _ x

yd_lt02 : Improper subunit

a,b: Misspelled parent name _ _ _ _ _ x

c,d: Parent unit does not specify subunit _ _ _ _ _ x

yd_lt03(a-c): Link with obsolete units _ _ _ _ _ x

yd_lt04a: Exhaust library disk quota _ _ _ _ x _

yd_lt05 : Omission of unreferenced subprograms

a: Subprograms never referenced _ _ _ _ x x

b: Isolated, inter-referenced subprograms _ _ _ _ x x

====================================================================

TOTAL "YES" and "PARTIAL" ANSWERS

FOR COMPILER ERROR DIAGNOSTICS _ _ _ _ _ _

DIVIDE

TOTAL BY PERCENTAGES

1 2 3 4 5 6

1. Is a diagnostic message printed? __ / 7 = _

2. Is the message in the general area

of difficulty? __ / 7 = _

3. Is the message at the correct

specific location? __ / 7 = _

4. Does the text of the message clearly

define the difficulty? __ / 7 = _

5. Is relevant non-local information

listed where appropriate? __ / 4 = _

6. Is error recovery appropriate? __ / 1 = _

ACES Version 2.0

DIAGNOSTIC ASSESSOR

RUNTIME ERRORS

TEST NAME DIAGNOSTIC QUESTIONS

====================================================================

1 2 3 4 5 6

====================================================================

yd_rt01a: Deadlock detection _ _ _ _ x x

yd_rt02 : Access errors

a: Dereference through deallocated types _ _ _ _ _ x

b: Different type object access _ _ _ _ _ x

c: Access through bad address _ _ _ _ _ x

d: Access through deallocated object _ _ _ _ _ x

yd_rt03a: Exhausting storage _ _ _ _ x _

yd_rt04 : Unhandled exception from main procedure

a: No exception handler _ _ _ _ x _

b: Unnamed RAISE statement _ _ _ _ x _

yd_rt05 : Uninitialized variables

a: Optimize _ _ _ _ x _

b: No optimize _ _ _ _ x _

===================================================================

TOTAL "YES" and "PARTIAL" ANSWERS

FOR COMPILER ERROR DIAGNOSTICS _ _ _ _ _ _

DIVIDE

TOTAL BY PERCENTAGES

1 2 3 4 5 6

1. Is a diagnostic message printed? __ / 10 = _

2. Is the message in the general area

of difficulty? __ / 10 = _

3. Is the message at the correct

specific location? __ / 10 = _

4. Does the text of the message clearly

define the difficulty? __ / 10 = _

5. Is relevant non-local information

listed where appropriate? __ / 4 = _

6. Is error recovery appropriate? __ / 5 = _