1.1 Identification

This Software Design Document (SDD) specifies the design of the Computer Software Configuration Item (CSCI) identified as the Support Software of the Ada Compiler Evaluation System (ACES) version 2.1. The design expressed in this SDD is based on the Ada Compiler Evaluation Capability (ACEC) Software Product Specification (SPS) Version 4.0 Support Software and the Software Design Document for the Support Software of the ACES Version 2.0. The requirements are derived from references [3] and [4] (see Section 2.0). This SDD describes the design for any software that has changed since the ACES Version 2.0. The reader is instructed to use the SPS, reference [1] and Version 2.0 of the SDD, reference [2] (see Section 2.0) for details concerning the design of the version 1.0 through 2.0 software.

1.2 System Overview

The purpose of the ACES is to help assess quality factors of Ada implementations. (An Ada implementation consists of a host computer, a target computer, operating systems for the two computers, and an Ada compilation system.) The ACES can provide compile-time performance data, run-time performance data, and system usability data. The ACES analysis programs can be used to compare the performance of two or more Ada implementations, or they can be used to indicate ways to get the best performance out of a particular implementation.

1.3 Document Overview

This SDD describes a partial design of the Support Software (CSCI 1) of the ACES. The design presented in this document describes the Support Software CSCI as composed of Computer Software Components (CSCs) and Computer Software Units (CSUs). This SDD provides a discussion of the Preliminary Design, Detailed Design, the CSCI Data and Data Files.

This SDD is organized according to the DI-MCCR-80012A Data Item Description, with some tailoring. The tailoring is discussed in the appropriate sections.

Section 1 - Scope - Identifies the CSCI, presents an overview of the CSCI, and describes the various sections.

Section 2 - Referenced Documents - Identifies all documents referenced in this SDD.

Section 3 - Preliminary Design - Describes the preliminary design of the ACES Version 2.1 Support Software.

Section 4 - Detailed Design - Describes the detailed design of the ACES Version 2.1 Support Software.

Section 5 - CSCI Data - Describes the global data within the ACES Version 2.1 Support Software and data passed between CSCs.

Section 6 - CSCI Data Files - Describes the files shared between the ACES Version 2.1 Support Software CSCs.

Section 7 - Requirements Traceability - Describes the environment from which the ACES Version 2.1 was developed.

Section 8 - Notes - Contains general information that aids in the understanding of this document. This section includes an alphabetical listing of all acronyms, abbreviations, and their meaning as used in this document.

2.0 REFERENCED DOCUMENTS

2.1 Government Documents

Document Document Number Title Ref #

D500-12626-1 Ada Compiler Evaluation Capability (ACEC) [1] Software Product Specification Version 4.0 Support Software, 5 April 1993

C0459 Software Design Document for the Support Software of the [2] ACES Version 2.0, November 22, 1994

C0594 Usability Enhancements Document for the ACES [3] Version 2.1, July 28, 1995

C0592 Operational Software Objectives Document for the ACES [4] Version 2.1, July 28, 1995

C0610 Software Design Document for the Operational Software of the ACES Version 2.1 (Draft), August 11, 1995. [5]

2.2 Non-Government Documents

There are no Non-Government documents referenced in this SDD.

3.0 PRELIMINARY DESIGN

To illustrate the ACES CSC interfaces it was necessary to arrive at a diagramming technique that handles generated code. Figure 3.0-1 illustrates the diagramming technique that is used in this SDD to describe the CSC interfaces (both control flow and data flow.) A solid box is used to depict a CSCI or a CSC. A dashed box is used to illustrate data being passed between CSCs. A solid box surrounded by a dashed box is used to illustrate source code or command scripts that have been generated by a CSC. A solid line is used to illustrate control flow, where one CSC will invoke (call) another CSC. An arrow points to the CSC being called. A dashed line is used to illustrate data flow. A data flow line is used to illustrate the transfer of data in the sense of either a data file or parameters. A data flow line is also used to illustrate the generation of source code, command scripts, and main programs.



Figure 3.0-1 CSC Interface Diagramming Key

3.1 CSCI Overview

Figure 3.1-1 provides an overview of the relationship between the ACES CSCIs.



Figure 3.1-1 ACES CSCI Overview

3.1.1 CSCI Architecture

Figure 3.1.1-1 illustrates the top-level architecture of the ACES Support Software CSCI. This section provides a brief overview of the purpose of each top-level CSC and provides a discussion of the interface between the top-level CSCs.



Figure 3.1.1-1 Support Software Top Level CSCI Architecture

The Pretest CSC coordinates the process of tailoring the ACES software for the implementation(s) being assessed. The Utilities CSC encapsulates the support software that is shared between CSCs and provides tests that verify some of the assessment techniques. The Harness CSC provides the capability of selecting and organizing assessment tests. The Analysis CSC provides for the conversion of test output logs which were formed by running the scripts made in the Harness into test result data files and some statistical processes that assess the implementation(s). The Quick-Look CSC is responsible for quickly organizing and running a subset of the ACES.

The Pretest CSC interfaces with the Utilities CSC in the following ways. The Utilities CSC provides the global declarations that the Pretest CSC shares with the other CSCs. The Pretest CSC coordinates the compiling of the Ada library units that make up the utilities. The Pretest CSC also coordinates the building and running of the tests that verify the assessment techniques.

The Pretest CSC interfaces with the Harness CSC in the following way. The Pretest CSC coordinates the compiling, building, and testing of the Harness executable.

The Pretest CSC interfaces with the Analysis CSC in the following way. The Pretest CSC coordinates the compiling of the Ada library units and building of the executables that make up the Analysis CSC. The Pretest CSC also coordinates the testing of the Analysis executables.

The Utilities CSC interfaces with the Harness CSC in the following way. The Utilities CSC provides the global declarations that the Harness CSC shares with the other CSCs.

The Utilities CSC interfaces with the Analysis CSC in the following way. The Utilities CSC provides the global declarations that the Analysis CSC shares with the other CSCs.

The Utilities CSC interfaces with the Quick-Look CSC in the following way. The Utilities CSC provides some declarations used by the Quick-Look CSC.



3.1.2 System States and Modes

The following design constraints are relevant to the system state and modes:

The ACES needs to provide the users with the ability to measure time using CPU time or elapsed time.

The ACES needs to provide the users with the ability to use either the "label'address" Ada construct in order to measure code size, or a user defined addressing construct.

The ACES needs to provide the users with the ability to use the ACES in an environment where executable size is limited.

The ACES needs to provide the users with the ability to setup their environment using automated tools or manually modify command script templates.

The user defines the mode in which the ACES is to run following the above design constraints.

3.1.3 Memory and Processing Time Allocation

The ACES is intended to run on a wide variety of Ada implementations and the memory and processing time allocations are part of what the ACES assesses. For this reason it is not possible to discuss memory and processing time allocations for the ACES.

3.2 CSCI Design Description

3.2.1 Pretest (CSC 1.1)

The Pretest is a logical abstraction for the process of preparing to run the ACES. This process was originally designed to be done manually through the use of editors and only after a great deal of background reading. The Pretest activity is automated and provides the users with the ability to get the ACES up and running on their system faster and easier. Default selections are to be provided for first time users to further facilitate the useability of the ACES. The Pretest activity consists of setup software and command scripts. The setup software prompts the users with questions concerning the operating system and compilation system for which they plan to operate the ACES. The responses to these questions are used to generate a series of command scripts that are used later in testing and running the Operation Software and the Analysis software. Sample scripts will be provided for a number of compiler versions and operating systems to further reduce the user's preparation time.

The design of the Pretest components has not been modified since version 2.0 and is therefore not presented in this document.

Table 3.2.1-1 identifies the requirements allocated to the Pretest CSC.

Table 3.2.1-1 Pretest Requirements

Requirement ID Requirement Text

SPT-P-6101
The Pretest software shall provide the user with the opportunity to exercise the same degree of detailed control of the pretest activity process that is provided by the Version 2.0 software.

SPT-P-6102
The number of questions presented to the user by the Pretest software shall be minimized by the use of default values and values that are specific to the operating system and compiler family being tested.

SPT-P-6103
The Pretest software shall base the default values for the following upon the operating system and compiler family being used. If no operating system or compiler family is identified, the user shall be required to supply the values.
* Command to execute a compiled program on the host system or on a self-hosted system;
* Command to delete a file;
* Suffix required for the delete command;
* Command to invoke a command script file;
* Command to copy a file;
* Command to rename a file;
* Symbol for the current directory;
* Symbol used as a separator in directory paths;
* Symbol used as the terminator of a directory path;
* Special first line of a script file (allow for multiple lines);
* Special symbol for the beginning of a script file line;
* Symbol or word to indicate the beginning of a comment line;
* Command to echo a string or value to the output file;
* Delimiter for a string being echoed to the output file;
* Suffix used to direct echoed string to a particular file;
* Symbol for the first parameter to a script file;
* Special form of the first parameter symbol for use in an echoed string;
* Symbol for the second parameter to a script file;
* Command to identify the Ada library for compiling to run on the host system (or a self-hosted system);
* Suffix expected for an Ada source code file;
* Command to compile a program (with default options) to be executed on the host system (or a self-hosted system);
* Command to compile (with full optimization) a program to be run on the target system (or a self-hosted system);
* Command to compile (with no optimization) a program to be run on the target system (or a self-hosted system);
* Command to compile (with full checking enabled) a program to be run on the target system (or a self-hosted system);
* Command to link a program to run on the target system (or a self-hosted system);
* Command to delete a compilation unit from the current Ada library;
* Command to delete the body (only) of a compilation unit from the current Ada library; and
* Command to delete the specification (only) of a compilation unit from the current Ada library.

SPT-P-6104
For commands and values that depend on the operating system and compiler family, the Pretest software shall make use of default responses applicable to the following combinations:

Operating System Compiler Family

DOS ThomSoft/Alsys
DOS GNAT
DOS Unspecified
UNIX ThomSoft/Alsys
UNIX Rational/VADS
UNIX GNAT
UNIX Unspecified
VMS VAX-Ada
VMS Unspecified
Unspecified ThomSoft/Alsys
Unspecified GNAT
Unspecified Rational/VADS

SPT-P-6105
The Pretest software shall provide instructions or shall refer to documentation containing instructions for providing user-defined defaults for the commands and values that are specified for the above (SPT-P-6104) combinations of operating system and compiler family.

SPT-P-6106
The Pretest software shall ask the user to provide the following information in all cases:
* The operating system being used (for the host system);
* The compiler family being used;
* Whether the system is for cross-compilation or is self-hosted;
* Whether the system implements Ada 83 or Ada 95; and
* The directory path for the "aces" root directory.

SPT-P-6107
The Pretest software shall assume the indicated values for the following:
* Execution time measurement (EL: Elapsed time)
* Compilation time measurement (EL: Elapsed time)
* Code size measurement technique (L: Label'Address)
* Host or self-hosted executable suffix (.exe)
* Bit size of largest integer type (32)
* Default file name extension for script files will be:
- DOS - .bat
- UNIX - .unx
- VMS - .com

SPT-P-6108
The Pretest software shall assume that the Portable math packages are to be used for the default setting if the system is an Ada 83 implementation; otherwise, it shall assume the existence of the math packages required by the Ada 95 standard.

SPT-P-6109
For cross-compilation systems only, the Pretest software shall require the user to provide the following commands and values:
* Command to define the Ada library for target system compilations;
* Command to compile a program to be run on the target system;
* Command to link a program to be run on the target system;
* Suffix for an executable file linked to run on the target system;
* Command to execute a compiled program on the target system;
* Command to delete the specification (only) of a unit from the target library;
* Command to delete the body (only) of a unit from the target library; and
* Command to delete both the specification and body of a unit from the target library.

SPT-P-6110
The Pretest software shall neither ask for nor use the following information that is requested by the Version 2.0 software:
* Command to execute a program under the debugger;
* Command to capture the status of a completed process;
* Command to export the status of a completing process;
* Command to compile with debugging enabled;
* Command to link a program for execution under the debugger;
* Method of managing the input and output files for the Include tool;
* Command for defining a logical symbol; and
* Operator used in defining a logical symbol.

SPT-P-6122
The support.zip file shall be revised to unload the initial files needed for Pretest into a "working" subdirectory of the "aces" directory.

SPT-P-6123
Pretest shall give the user the opportunity to modify the default as well as the previously constructed environment file by rerunning Setup.

SPT-P-6124
Setup shall be modified so that the Ada source files required for Pretest will be copied from the distribution subdirectories to the "source" subdirectory where they can be accessed for compilation. All other required files shall be copied from the distribution directories (note SPT-D-8109) to "working". This will be done at the beginning of the Pretest activity and all files will be renamed at that point if needed.

SPT-P-8107
Setup shall inform the user which (83/95) compiler version is selected in any existing environment file. It now only informs the user of the selection of cross/self compiler.

SPT-P-8116
Setup shall make a backup copy of any existing environment file before it offers the user a chance to make changes. It shall also offer the user the chance to revert to the old version by replacing quit with two choices: (1) generate new file with the new selections; or (2) exit and save none of the changes; go back to the old version.

SPT-P-8117
The Pretest software unit "zg_incld" (four versions) shall specify the type CHARACTER for at least one of the literals in the code "for i in 'a' .. 'z' loop".



3.2.2 Utilities (CSC 1.2)

The Utilities CSC is a logical abstraction for the software that provides a variety of functions to the other CSCs. The utility software consists of utility tests, initialization software, inclusion software, global type and object declarations, and mathematics software. The utility tests enable the system to check the system's ability to use a cpu clock, and measure code size. The initialization software builds baselined timing files that are used during the building of the operational software. The inclusion software copies the appropriate timing code into the performance test templates, producing the actual performance tests. Library units that provide global types and shared objects are necessary for the compiling of the various CSCs. The mathematics software provides the software necessary to perform statistical analysis.

The design of this CSC has not been modified since version 2.0 and therefore is not documented here.

3.2.3 Harness (CSC 1.3)

The HARNESS allows the users to select performance tests individually, by subgroup, and by group. Harness also allows the user to generate command files to compile, link, and run the selected tests, and to monitor the results. A new CSU is to be added to the Harness CSC to allow the user to choose performance tests by identifying performance issues. The user will be able to choose tests through performance issues and then modify the individual test selection, or the user may continue to use the HARNESS in the same manner as was allowed in previous versions of the software.

The use of the HARNESS is optional, but it is expected that all ACES users who run performance tests will use the HARNESS if their system permits. The basic design of the Harness software has been modified by adding this new CSU. The design changes for Harness are detailed in Section 4.3.

Table 3.2.3-1 identifies the requirements allocated to the Harness CSC.

Table 3.2.3-1 Harness Requirements

Requirement ID Requirement Text

OPS-T-5301
The ACES shall determine the performance characteristics of conversions between enumeration or numeric (integer, modular, float, fixed, decimal) types and String, using attributes and instantiations of the generic IO packages in Text_IO.

OPS-T-5302
The ACES shall determine the performance characteristics of the "Succ" and "Pred" attributes for real types.

OPS-T-5303
The ACES shall determine the performance of a subprogram call when the subprogram is passed to the calling subprogram via an access-to-subprogram parameter.

OPS-T-5304
The ACES shall determine the performance characteristics of decimal type operations (if supported).

OPS-T-5305
The ACES shall determine the performance impact of referencing and assigning to an object designated by a subprogram's formal access parameter.

OPS-T-5306
The ACES shall determine the performance of assignment of controlled types.

OPS-T-5307
The ACES shall determine the performance of user-defined finalization when exiting the scope of a controlled object.

OPS-T-5308
The ACES shall determine the time required for a call on a protected operation in the simplest of cases.

OPS-T-5309
The ACES shall determine the performance characteristics of the requeue statement.

OPS-T-5310
The ACES shall determine the performance characteristics of the delay until statement.

OPS-T-5311
The ACES shall investigate the performance characteristics of asynchronous transfer of control in various scenarios.

OPS-T-5312
The ACES shall determine the performance characteristics of references to entities in child packages.

OPS-T-5313
The ACES shall determine the performance characteristics of the operations in package Ada.Exceptions.

OPS-T-5314
The ACES shall determine the performance characteristics of the address arithmetic operations declared in the package System.Storage_Elements.

OPS-T-5315
The ACES shall determine the performance characteristics of the conversion operations declared in the package System.Address_To_Access_Conversions.

OPS-T-5316
The ACES shall determine the performance characteristics of the storage management operations declared in the package System.Storage_Pools.

OPS-T-5317
The ACES shall determine the performance characteristics of passing data among units by using the stream operations given by attributes.

OPS-T-5318
The ACES shall determine the performance characteristics of the operations declared in package Strings.Fixed.

OPS-T-5319
The ACES shall determine the performance characteristics of the operations declared in package Strings.Bounded.

OPS-T-5320
The ACES shall determine the performance characteristics of the operations declared in package Strings.Unbounded.

OPS-T-5321
The ACES shall determine the performance characteristics of the operations declared in Ada.Numerics.Elementary_Functions and in an instantiation of Ada.Numerics.Generic_Elementary_Functions, using Float.

OPS-T-5322
The ACES shall determine the performance characteristics of the random value function declared in instantiations of Ada.Numerics.Discrete_Random.

OPS-T-5323
The ACES shall determine the performance characteristics of the shift and rotate operations provided by package Ada.Interfaces.

OPS-T-5324
The ACES shall determine the performance characteristics of calls to subprograms written in C, C++, Fortran, and COBOL, as supported by the specific implementation.

OPS-T-5325
The ACES shall determine the performance characteristics of the operations provided by package Ada.Interfaces.C.

OPS-T-5326
The ACES shall determine the performance characteristics of the operations provided by package Ada.Interfaces.C.Strings.

OPS-T-5327
The ACES shall determine the performance characteristics of the operations provided by package Ada.Interfaces.C.Pointers.

OPS-T-5328
The ACES shall determine the performance characteristics of the operations provided by package Ada.Interfaces.COBOL.

OPS-T-5329
The ACES shall determine the performance characteristics of the operations provided by package Ada.Interfaces.Fortran.

OPS-T-5401
The ACES shall determine the performance impact of using aliased objects as subprogram parameters.

OPS-T-5402
The ACES shall determine the performance effects of declaring objects of unconstrained types (the initial constraint being taken from the initialization expression).

OPS-T-5403
The ACES shall determine whether a performance cost is incurred when referencing an extension component of a tagged type.

OPS-T-5404
The ACES shall determine whether a performance cost is incurred in referencing and updating arrays and records whose components are aliased.

OPS-T-5405
The ACES shall determine the relative performance of dispatching for various kinds of controlling operands.

OPS-T-5406
The ACES shall determine the performance assignments to an aliased object and evaluations of an aliased object.

OPS-T-5407
The ACES shall determine the performance impact of using general access types to reference data at different accessibility levels.

OPS-T-5408
The ACES shall determine the performance characteristics of evaluations of the Access attribute.

OPS-T-5409
The ACES shall determine the relative performance of elaboration of private type and non-private type declarations.

OPS-T-5410
The ACES shall determine the performance impact of using discriminants for tasks and for protected objects.

OPS-T-5411
The ACES shall determine the performance impact of using exception handlers in accept bodies.

OPS-T-5412
The ACES shall determine the performance characteristics of exception propagation from protected subprograms and entries of protected objects.

OPS-T-5413
The ACES shall determine the performance impact of the use of representation clauses that have been added to the language.

OPS-T-5414
The ACES shall determine the performance characteristics of evaluations of the Bit_Order attribute.

OPS-T-5415
The ACES shall determine the performance impact of using unchecked conversion to produce valid and invalid values of a type.

OPS-T-5416
The ACES shall determine the performance impact of using the Unchecked_Access attribute to create access values.

OPS-T-5417
The ACES shall determine the performance characteristics of evaluations of the Storage_Pool and Storage_Size attributes for access subtypes.

OPS-T-5418
The ACES shall determine the performance characteristics of the operations declared in package Ada.Characters.Handling.

OPS-T-5419
The ACES shall determine the performance characteristics of Wide_Character operations.

OPS-T-5420
The ACES shall determine the performance characteristics of Wide_String operations.

OPS-T-5421
The ACES shall determine the performance characteristics of conversions between enumeration or numeric (integer, modular, float, fixed, decimal) types and Wide_String, using attributes and instantiations of the generic IO packages in Wide_Text_IO.

OPS-T-5422
The ACES shall determine the performance characteristics of the operations declared in package Strings.Wide_Fixed.

OPS-T-5423
The ACES shall determine the performance characteristics of the operations declared in package Strings.Wide_Bounded.

OPS-T-5424
The ACES shall determine the performance characteristics of the operations declared in package Strings.Wide_Unbounded.

OPS-T-5425
The ACES shall determine the performance characteristics of the operations declared in package Streams.Stream_IO. Compare to the performance of equivalent attributes after applying the Stream function to create a Stream_Access value from the file.

OPS-T-7101
The following tests shall be renamed:

a9_ms_atc_taken to tk_at_atc_taken_01
a9_ms_atc_not_taken to tk_at_atc_not_taken_02
a9_ms_formal_acc_subp_01 to gn_su_formal_acc_subp_01
a9_ms_formal_package_01 to gn_su_formal_package_01
a9_ms_modular_oper_01 to ar_mt_modular_oper_01
a9_ms_modular_oper_02 to ar_mt_modular_oper_02
a9_ms_random_01 to ar_rn_random_float_01

OPS-T-7102
The following tests shall be renamed:

a9_ob_class_wide_dynamic_01 to oo_dp_class_wide_dynamic_01
a9_ob_class_wide_static_01 to oo_dp_class_wide_static_01
a9_ob_parms_01_deriv to oo_tx_parms_deriv_01
a9_ob_parms_05_deriv to oo_tx_parms_deriv_05
a9_ob_parms_10_deriv to oo_tx_parms_deriv_10
a9_ob_ext_aggregate_01 to oo_tx_ext_aggregate_01
a9_ob_ext_aggregate_02 to oo_tx_ext_aggregate_02
a9_ob_simp_aggregate_01 to oo_tx_simp_aggregate_01
a9_ob_simp_aggregate_02 to oo_tx_simp_aggregate_02

OPS-T-7103
The following tests shall be renamed:

a9_pt_prot_access_01 to pt_po_prot_access_01
a9_pt_prot_access_02 to pt_po_prot_access_02
a9_pt_prot_access_03 to pt_po_prot_access_03

SPT-H-5101
The Harness shall recognize new performance test groups and subgroups, as determined by new and renamed performance tests.

SPT-H-5102
The Harness shall recognize names of new default main programs that incorporate new and renamed tests.

SPT-H-5103
Main programs that incorporate new or renamed tests shall not include any previously existing tests.

SPT-H-5104
The Harness shall display all new test names that are appropriate for the implementation being tested.

SPT-H-5105
The Harness shall display tests specific to Ada 95 only if the user has identified the implementation as an Ada 95 implementation.

SPT-H-5106
The Harness shall display group and subgroup names only if the groups and subgroups contain tests applicable to the language version of the implementation under test.

SPT-H-6201
The Harness software shall provide the user an option of selecting tests based on the performance issues of interest to the user.

SPT-H-6202
The Harness shall provide the user with the option of bypassing the performance issues menu and proceeding directly to the Harness menu.

SPT-H-6203
The Harness shall provide a default set of performance issues, which shall be preselected when the performance issues menu first appears.

SPT-H-6204
The Harness shall allow the user to select or de-select any of the performance issues.

SPT-H-6205
The Harness shall provide at least the following options on the performance issues menu: - select all performance issue selections; - deselect all performance issue selections; - request help; and - quit the performance issues menu (exit to the Groups menu).

SPT-H-6206
If the performance issues menu is not used, the ACES shall work in the same manner as in ACES, Version 2.0. If a test.sel file exists, Harness shall use that as the initial test selection. Otherwise, no tests will be selected when the Harness menu is initially displayed.

SPT-H-7201
The Harness shall provide a mechanism for selecting User-Defined benchmarks in a fashion similar to the way that tests are currently handled in the ACES.

SPT-H-7202
The Harness shall provide a mechanism for generating command scripts that will execute User-Defined benchmarks in a fashion similar to the way that command scripts are currently generated by the Harness.

SPT-H-8201
The Harness software shall indicate, in its error messages in UPDATING, the name of the file causing the error rather than the name of the file following the file that caused the error.

3.2.4 Analysis (CSC 1.4)

The Analysis CSC is a logical abstraction for the processes involved in analyzing the test results generated in the operational software. The Analysis CSC is composed of the Condense process, Comparative Analysis process and the Single System Analysis process. The basic design for these components has not been modified since version 2.0 and therefore the design for this CSC is not presented in this document.

Table 3.2.4-1 identified the requirements allocated to the Analysis CSC.

Table 3.2.4-1 Analysis Requirements

Requirement ID Requirement Text

OPS-T-5301
The ACES shall determine the performance characteristics of conversions between enumeration or numeric (integer, modular, float, fixed, decimal) types and String, using attributes and instantiations of the generic IO packages in Text_IO.

OPS-T-5302
The ACES shall determine the performance characteristics of the "Succ" and "Pred" attributes for real types.

OPS-T-5303
The ACES shall determine the performance of a subprogram call when the subprogram is passed to the calling subprogram via an access-to-subprogram parameter.

OPS-T-5304
The ACES shall determine the performance characteristics of decimal type operations (if supported).

OPS-T-5305
The ACES shall determine the performance impact of referencing and assigning to an object designated by a subprogram's formal access parameter.

OPS-T-5306
The ACES shall determine the performance of assignment of controlled types.

OPS-T-5307
The ACES shall determine the performance of user-defined finalization when exiting the scope of a controlled object.

OPS-T-5308
The ACES shall determine the time required for a call on a protected operation in the simplest of cases.

OPS-T-5309
The ACES shall determine the performance characteristics of the requeue statement.

OPS-T-5310
The ACES shall determine the performance characteristics of the delay until statement.

OPS-T-5311
The ACES shall investigate the performance characteristics of asynchronous transfer of control in various scenarios.

OPS-T-5312
The ACES shall determine the performance characteristics of references to entities in child packages.

OPS-T-5313
The ACES shall determine the performance characteristics of the operations in package Ada.Exceptions.

OPS-T-5314
The ACES shall determine the performance characteristics of the address arithmetic operations declared in the package System.Storage_Elements.

OPS-T-5315
The ACES shall determine the performance characteristics of the conversion operations declared in the package System.Address_To_Access_Conversions.

OPS-T-5316
The ACES shall determine the performance characteristics of the storage management operations declared in the package System.Storage_Pools.

OPS-T-5317
The ACES shall determine the performance characteristics of passing data among units by using the stream operations given by attributes.

OPS-T-5318
The ACES shall determine the performance characteristics of the operations declared in package Strings.Fixed.

OPS-T-5319
The ACES shall determine the performance characteristics of the operations declared in package Strings.Bounded.

OPS-T-5320
The ACES shall determine the performance characteristics of the operations declared in package Strings.Unbounded.

OPS-T-5321
The ACES shall determine the performance characteristics of the operations declared in Ada.Numerics.Elementary_Functions and in an instantiation of Ada.Numerics.Generic_Elementary_Functions, using Float.

OPS-T-5322
The ACES shall determine the performance characteristics of the random value function declared in instantiations of Ada.Numerics.Discrete_Random.

OPS-T-5323
The ACES shall determine the performance characteristics of the shift and rotate operations provided by package Ada.Interfaces.

OPS-T-5324
The ACES shall determine the performance characteristics of calls to subprograms written in C, C++, Fortran, and COBOL, as supported by the specific implementation.

OPS-T-5325
The ACES shall determine the performance characteristics of the operations provided by package Ada.Interfaces.C.

OPS-T-5326
The ACES shall determine the performance characteristics of the operations provided by package Ada.Interfaces.C.Strings.

OPS-T-5327
The ACES shall determine the performance characteristics of the operations provided by package Ada.Interfaces.C.Pointers.

OPS-T-5328
The ACES shall determine the performance characteristics of the operations provided by package Ada.Interfaces.COBOL.

OPS-T-5329
The ACES shall determine the performance characteristics of the operations provided by package Ada.Interfaces.Fortran.

OPS-T-5401
The ACES shall determine the performance impact of using aliased objects as subprogram parameters.

OPS-T-5402
The ACES shall determine the performance effects of declaring objects of unconstrained types (the initial constraint being taken from the initialization expression).

OPS-T-5403
The ACES shall determine whether a performance cost is incurred when referencing an extension component of a tagged type.

OPS-T-5404
The ACES shall determine whether a performance cost is incurred in referencing and updating arrays and records whose components are aliased.

OPS-T-5405
The ACES shall determine the relative performance of dispatching for various kinds of controlling operands.

OPS-T-5406
The ACES shall determine the performance assignments to an aliased object and evaluations of an aliased object.

OPS-T-5407
The ACES shall determine the performance impact of using general access types to reference data at different accessibility levels.

OPS-T-5408
The ACES shall determine the performance characteristics of evaluations of the Access attribute.

OPS-T-5409
The ACES shall determine the relative performance of elaboration of private type and non-private type declarations.

OPS-T-5410
The ACES shall determine the performance impact of using discriminants for tasks and for protected objects.

OPS-T-5411
The ACES shall determine the performance impact of using exception handlers in accept bodies.

OPS-T-5412
The ACES shall determine the performance characteristics of exception propagation from protected subprograms and entries of protected objects.

OPS-T-5413
The ACES shall determine the performance impact of the use of representation clauses that have been added to the language.

OPS-T-5414
The ACES shall determine the performance characteristics of evaluations of the Bit_Order attribute.

OPS-T-5415
The ACES shall determine the performance impact of using unchecked conversion to produce valid and invalid values of a type.

OPS-T-5416
The ACES shall determine the performance impact of using the Unchecked_Access attribute to create access values.

OPS-T-5417
The ACES shall determine the performance characteristics of evaluations of the Storage_Pool and Storage_Size attributes for access subtypes.

OPS-T-5418
The ACES shall determine the performance characteristics of the operations declared in package Ada.Characters.Handling.

OPS-T-5419
The ACES shall determine the performance characteristics of Wide_Character operations.

OPS-T-5420
The ACES shall determine the performance characteristics of Wide_String operations.

OPS-T-5421
The ACES shall determine the performance characteristics of conversions between enumeration or numeric (integer, modular, float, fixed, decimal) types and Wide_String, using attributes and instantiations of the generic IO packages in Wide_Text_IO.

OPS-T-5422
The ACES shall determine the performance characteristics of the operations declared in package Strings.Wide_Fixed.

OPS-T-5423
The ACES shall determine the performance characteristics of the operations declared in package Strings.Wide_Bounded.

OPS-T-5424
The ACES shall determine the performance characteristics of the operations declared in package Strings.Wide_Unbounded.

OPS-T-5425
The ACES shall determine the performance characteristics of the operations declared in package Streams.Stream_IO. Compare to the performance of equivalent attributes after applying the Stream function to create a Stream_Access value from the file.

OPS-T-7101
The following tests shall be renamed:

a9_ms_atc_taken to tk_at_atc_taken_01
a9_ms_atc_not_taken to tk_at_atc_not_taken_02
a9_ms_formal_acc_subp_01 to gn_su_formal_acc_subp_01
a9_ms_formal_package_01 to gn_su_formal_package_01
a9_ms_modular_oper_01 to ar_mt_modular_oper_01
a9_ms_modular_oper_02 to ar_mt_modular_oper_02
a9_ms_random_01 to ar_rn_random_float_01

OPS-T-7102
The following tests shall be renamed:

a9_ob_class_wide_dynamic_01 to oo_dp_class_wide_dynamic_01
a9_ob_class_wide_static_01 to oo_dp_class_wide_static_01
a9_ob_parms_01_deriv to oo_tx_parms_deriv_01
a9_ob_parms_05_deriv to oo_tx_parms_deriv_05
a9_ob_parms_10_deriv to oo_tx_parms_deriv_10
a9_ob_ext_aggregate_01 to oo_tx_ext_aggregate_01
a9_ob_ext_aggregate_02 to oo_tx_ext_aggregate_02
a9_ob_simp_aggregate_01 to oo_tx_simp_aggregate_01
a9_ob_simp_aggregate_02 to oo_tx_simp_aggregate_02

OPS-T-7103
The following tests shall be renamed:

a9_pt_prot_access_01 to pt_po_prot_access_01
a9_pt_prot_access_02 to pt_po_prot_access_02
a9_pt_prot_access_03 to pt_po_prot_access_03

SPT-A-5201
The Analysis software shall recognize the names of new performance test groups, subgroups, tests, and default main programs.

SPT-A-5202
The Menu program shall allow selection of groups consisting entirely of Ada 95 tests if and only if at least one selected implementation has been identified as an Ada 95 implementation.

SPT-A-5203
The Comparative Analysis program shall incorporate results from new performance test groups, subgroups, default main programs, and tests into its analyses and reports.

SPT-A-5204
The Single-System Analysis program shall provide comparisons of sets of new tests and of sets containing both new and previously existing tests as appropriate.

SPT-A-6301
The Analysis software shall provide default reports which shall be designed to provide the user with the least amount of information which would also be usable. The first CA report shall present the System Factors for all test groups. The second shall present the number of tests successfully implemented.

SPT-A-6302
The default metric for Comparative Analysis reports shall be execution time.

SPT-A-6303
The Analysis software shall provide default reports which shall be designed to provide the user with the least amount of information which would also be usable. The SSA report shall present a ratio of the smallest to the largest reported Language Features for each test comparison.

SPT-A-7301
The ACES shall provide a method to perform Comparative Analysis on User-Defined benchmarks.

SPT-A-7302
The ACES shall provide a method to perform Single-System analysis on User-Defined benchmarks.

SPT-A-8301
The Menu software shall allow the user to advance to the next screen without having to type "n" in addition to the command where multiple selections before going to the next screen are not possible.

SPT-A-8302
The Analysis software, on the MAIN MENU screen, shall tell the user to select a command or a tool and a command (separating them with a comma) instead of telling the user to "select one tool separated by a comma."

SPT-A-8303
The Analysis software shall be designed to exit gracefully if multiple errors come up after all selections are made instead of having the user resort to a keyboard abortion command.

SPT-A-8304
The Analysis software shall be designed to give adequate error messages and exit gracefully if it is missing a file so that the user will know what files are required for it to function properly. The software shall indicate if the file is missing or if it cannot locate the name it was given to find and where to make that change.

SPT-A-8305
The Analysis software shall be redesigned so that the GROUPS menu either fits on one screen or appears on two screens in a way that is easy to use. Presently the title and "1. Applications" scroll off screen when Ada_95 is an option.

SPT-A-8306
The Analysis software shall be redesigned so that the RUN OR SAVE REQUEST menu will indicate in all cases either that no reports were chosen (and therefore no output is to be expected) or what has been chosen and what will be produced.

SPT-A-8307
The Analysis software shall be revised so that when a screen indicates that one of a group of items must be selected, then the user will really have to make a selection before going to the next screen.

This occurs on at least the following screens: - SUMMARY OF ALL GROUPS LEVEL REPORT OPTIONS: - SSA MAIN REPORT OPTIONS - SSA HLS REPORT OPTIONS

SPT-A-8308
The Analysis software shall present a new menu that is a combination of the present MAIN and INITIAL menus.

SPT-A-8311
The Analysis software menus shall use language consistent with the actual reports. This shall include but not be limited to: - Selection 7 of the SUMMARY OF ALL GROUPS LEVEL REPORT OPTIONS, which indicates on the screen "errors" and on the report "successes".

SPT-D-8309
The Analysis software shall define the help menu for BUILD/MODIFY SYSTEM NAME FILE (GENERAL INFORMATION) so that the file names explained there are the same as those that appear in selections and make other corrections as needs so that the help menus correspond to the screens they serve.



3.2.5 Quick-Look (CSC 1.5)

The Quick-Look CSC provides the users with the ability to quickly setup and run a subset of the ACES.

The design of the Quick-Look components has not been modified since version 2.0 and is therefore not presented in this document.

Table 3.2.5-1 identifies the requirements allocated to the Quick-Look CSC.

Table 3.2.5-1 Quick-Look Requirements

Requirement ID Requirement Text

SPT-Q-6111
The Quick-Look software shall provide the user with the opportunity to exercise the same degree of detailed control of the Quick-Look activity process that is provided by the Version 2.0 software.

SPT-Q-6112
The number of questions presented to the user by the Quick-Look software shall be minimized by the use of default values and values that are specific to the operating system and compiler family being tested.

SPT-Q-6113
The Quick-Look software shall base the default values for the following upon the operating system and compiler family being used. If no operating system or compiler family is identified, the user shall be required to supply the values.
* Command to execute a compiled program on the host system or on a self-hosted system;
* Command to delete a file;
* Suffix required for the delete command;
* Command to invoke a command script file;
* Command to copy a file;
* Command to rename a file;
* Symbol for the current directory;
* Symbol used as a separator in directory paths;
* Symbol used as the terminator of a directory path;
* Special first line of a script file (allow for multiple lines);
* Special symbol for the beginning of a script file line;
* Symbol or word to indicate the beginning of a comment line;
* Command to echo a string or value to the output file;
* Delimiter for a string being echoed to the output file;
* Suffix used to direct echoed string to a particular file;
* Symbol for the first parameter to a script file;
* Special form of the first parameter symbol for use in an echoed string;
* Symbol for the second parameter to a script file;
* Command to identify the Ada library for compiling to run on the host system (or a self-hosted system);
* Suffix expected for an Ada source code file;
* Command to compile a program (with default options) to be executed on the host system (or a self-hosted system);
* Command to compile (with full optimization) a program to be run on the target system (or a self-hosted system);
* Command to link a program to run on the target system (or a self-hosted system); and
* Command to delete a compilation unit from the current Ada library.

SPT-Q-6114
For commands and values that depend on the operating system and compiler family, the Quick-Look software shall make use of default responses applicable to the following combinations: Operating System Compiler Family DOS ThomSoft/Alsys
DOS GNAT
DOS Unspecified
UNIX ThomSoft/Alsys
UNIX Rational/VADS
UNIX GNAT
UNIX Unspecified
VMS VAX-Ada
VMS Unspecified
Unspecified ThomSoft/Alsys
Unspecified GNAT
Unspecified Rational/VADS

SPT-Q-6115
The Quick-Look software shall provide instructions or shall refer to documentation containing instructions for providing user-defined defaults for the commands and values that are specified for the above combinations of operating system and compiler family.

SPT-Q-6116
The Quick-Look software shall ask the user to provide the following information in all cases:
* The operating system being used (for the host system);
* The compiler family being used;
* Whether the system is for cross-compilation or is self-hosted;
* Whether the system implements Ada 83 or Ada 95; and
* The directory path for the "aces" root directory.

SPT-Q-6117
The Quick-Look software shall assume the indicated values for the following:
* Host or self-hosted executable suffix (.exe)
* Bit size of largest integer type (32)
* Default file name extension for script files will be: - DOS - .bat - UNIX - .unx - VMS - .com

SPT-Q-6118
For cross-compilation systems only, the Quick-Look software shall require the user to provide the following commands and values:
* Command to define the Ada library for target system compilations;
* Command to compile a program to be run on the target system;
* Command to link a program to be run on the target system;
* Suffix for an executable file linked to run on the target system;
* Command to execute a compiled program on the target system;
* Command to delete the specification (only) of a unit from the target library;
* Command to delete the body (only) of a unit from the target library; and
* Command to delete both the specification and body of a unit from the target library.

SPT-Q-6119
The Quick-Look software shall neither ask for nor use the following information that is requested by the Version 2.0 software:
* Method of managing the input and output files for the Include tool;
* Command for defining a logical symbol; and
* Operator used in defining a logical symbol.

SPT-Q-6127
Setup shall be modified so that the Ada source files required for Quick-Look will be copied from the distribution subdirectories to the "source" subdirectory where they can be accessed for compilation. All other required files shall be copied from the distribution directories (note SPT-D-8109) to "ql_work". This will be done at the beginning of the Quick-Look activity and all files will be renamed at that point if needed.

SPT-Q-6130
The support.zip file shall be revised to unload the initial files needed for Quick-Look into a "ql_work" subdirectory of the "aces" directory.

SPT-Q-7101
The Quick-Look facility shall be flexible enough to allow additional User-Defined benchmark tests to be added.

SPT-Q-7102
The Quick-Look facility shall be able to report on the results of additional User- Defined benchmark tests.

SPT-Q-8402
The Quick-Look software unit "zg_incld" shall specify the type CHARACTER for at least one of the literals in the code "for i in 'a' .. 'z' loop".

4.0 DETAILED DESIGN

4.1 Pretest (CSC 1.1)

Figure 4.1-1 illustrates the Pretest architecture.



Figure 4.1-1 Pretest (CSC 1.1) Architecture

4.1.1 Setup (CSC 1.1.1)

The CSUs for this CSC are Identify Environment Data and Generate Scripts.

The design for Setup has not been modified since version 2.0. The only changes made are to accommodate the new tests for Ada 95 and the use of default choices for users.

4.1.1.1 Identify Environment Data (CSU 1.1.1.1)

The design for this CSU has not been modified since version 2.0. The only changes are those made to accommodate the use of default choices for users.

4.1.1.2 Generate Scripts (CSU 1.1.1.2)

The design for this CSU has not been modified since version 2.0. The only changes are those made to accommodate the use of default choices for users.

4.1.2 Scripts (CSC 1.1.2)

One script has been added since ACES Version 2.0. That script is for Pretest Step 0. It has been added to make the system more automated.

4.1.2.1 Scripts (CSU 1.1.2.1)

Purpose: The script that was added copies all of the necessary files from the distribution subdirectories automatically into the working directory, with the correct suffixes.

Requirements: See Section 3.2.1.

Design Constraints: The script is applicable to a specific Ada implementation (a specific operating system and a specific Ada compiler vendor). Generation of the script is driven by a non-specific template and by compiler and operating system data supplied by the user or by default data files.

CHI: None.

Rationale for Other Language: Requires command level interface to the operating system.

IO Data Elements: None. Algorithm: Copy Pretest files from the distribution to the working subdirectories. If file extension is different from system file extension Change to system file extension end if end

Error Handling: None.

Other Units: None.

Invoked By: User.

Calls: None.

Data Structures: None.

Data Files: A new template file shall be designed to perform the copying of the files.

4.2 Utilities (CSC 1.2)

Figure 4.2-1 illustrates the Utilities architecture.



Figure 4.2-1 Utilities (CSC 1.2) Architecture

The design for the Utilities CSC has not been modified since version 2.0.

4.3 Harness (CSC 1.3)

The Harness design has been modified by adding a CSU to allow performance test selections through the identification of relevant performance issues. The user has the option of either using performance issues to select tests or selecting tests in the previous manner by group, subgroup, and test names. The user may also use a combination of the two methods by identifying performance issues and then modifying the test selection.

The Select Performance Issues CSU has been added to the Harness CSC and is the only design change since ACES 2.0.

4.3.1 Select Performance Issues (CSU 1.3.1)

Purpose: Generate a new menu to allow the user to select performance tests by identifying performance issues. These issues will identify tests that are automatically selected for processing. After selecting performance issues important to the user's application, the user will run the Harness menu to modify the test selection and to build scripts to run the performance tests.

Design Constraints: The user should be able to operate the Harness in the same manner as previously allowed; the user will still be able to select tests by group, subgroup and test.

Default performance issues will be assigned when the user first enters the Select Performance Issues menu.

Help text will be available.

The Select Performance Issue menu will provide options to select all performance issues and to clear all selections.

CHI: The initial use screen will allow the user to identify how he/she wants to identify performance tests. The user may identify performance issues important to the user's application or may enter the harness screen that allows the user to select groups/subgroups, and individual tests.

Additional menus will be displayed to allow the user to identify performance issues. The menus will allow the following options: default performance issues will initially be selected
the user can select or de-select any performance issue
the user can get help information
the user can select all performance issues
the user can clear all performance issues
the user can look at the next or previous screen of performance issues
the user can quit Harness
the user can proceed to the Harness main menu

Rationale for other Language: N/A

IO Data Elements: Input data elements: Test selection data structure.

Output data elements: Test selection data structure.
Quit request.

Algorithm:

Modify zh_hrn15.ada.

Add a call in the first line of the harness procedure (after the begin) to call a new routine that handles the performance issue selection.

Surround the existing harness code in an if statement (after the begin statement and call to the new routine).

begin
Select_Performance_Issues (Test_selection_data_structure,    -- inout
                           Quit_Request);                    -- out
       if Not Quit_Request
       then
         ... existing code ...
       end if;

-- the basic design of the performance issue selection routine is:
           BEGIN -- predefined selection routine
              Set Quit_Request = FALSE
              Put up Harness_Entry screen.
              Get user_selects_harness_tests input.
              If user_selects_harness_tests
              then
                exit procedure
              elsif -- select set of tests based on performance issues
                loop
                  read performance_issue and default_PI from zh_prfmi.txt
                  if performance_issue = blank
                  then
                    exit loop
                  end if
                  if default_PI
                  then
                    set performance_issue selected
                  end if
                end loop
                Put up Performance_Issues_Menu.
                loop
                  Get Performance_Issues_Menu_input.
                  if Performance_Issues_Menu_input = Help
                    CALL Performance_Issues_Menu_Help.
                  elsif Performance_Issues_Menu_input = Quit
                    set Quit_Request = TRUE
                    exit procedure
                  elsif Performance_Issues_Menu_input = Next Screen
                    Put up next Performance_Issue_Menu.
                  elsif Performance_Issues_Menu_input = Previous Screen
                    Put up Previous Performance_Issue_Menu.
                  elsif Performance_Issues_Menu_input = Harness Menu
                    current_group_name := unassigned
                    loop
                      read test_name and performance_issue from zh_pfrmi.txt
                      if eof then exit loop
                      if performance issue is selected
                      then
                        if group_name (of_test) = current_group_name
                          if Ada95Compiler
                            set test selected
                            if Ada95Test
                              set test selected
                            end if
                          end if
                        else
                          read group_name (data_file) into memory
                          if Ada95Compiler
                            set test selected
                            if Ada95Test
                              set test selected
                            end if
                          end if
                        end if  -- group
                      end if  --performance issue selected
                    end loop
                    Write test.sel file
                    exit procedure
                  elsif Performance_Issues_Menu_input = Clear All
                    Set All Performance Issues to NOT_SELECTED
                  elsif Performance_Issues_Menu_input = Select All
                    Set All Performance Issues to SELECTED
                  elsif Performance_Issues_Menu_input =
                            positive or negative performance_issue
                    modify performance_issues data structure.
                    put plus (+) or minus (-) next to appropriate
                            performance_issues when re-displaying menu
                  else
                    print input error message
                    prompt to continue
                    re-display menu
                  end if
                end loop
              else  -- Quit
                Set Quit_Request = TRUE
                exit procedure
              end if
           END -- performance issue selection routine

Error Handling:  The CSU will handle invalid user menu responses and re-display 
menu and re-prompt user for new input.

Other Units: None.

Invoked By: Harness main procedure.

Calls: None.

Data Structures: Uses existing Test_selection_data_structure (TestRecord in Package GlobalPackage) with one additional data element that identifies a test as an Ada83 or an Ada95 test (Ada95Test : BOOLEAN).

Local Files: An additional input file will be created and will be input to the Harness software. The structure and data contained in this file is described below:

File Name:
zh_prfmi.txt text file
Data:
Performance_Issue #1 default_PI
Performance_Issue #2
Performance_Issue #3
Performance_Issue #4 default_PI
...
Performance_Issue_#n-1
Performance_Issue_#n default_PI
blank line
test_name Performance_Issue_#i
test_name Performance_Issue_#j
...
test_name Performance_Issue_#i
test_name Performance_Issue_#k
(end of file)

Test names above will be sorted by group.

4.4 Analysis (CSC 1.4)

Figure 4.4-1 illustrates the Analysis architecture.



Figure 4.4-1 Analysis (CSC 1.4) Architecture

The design for the Analysis has not been modified since version 2.0. The software components that make up this CSC have not been changed since version 2.0 and their design is not repeated in this document.

4.5 Quick-Look (CSC 1.5)

Figure 4.5-1 illustrates the Quick-Look architecture.



Figure 4.5-1 Quick-Look (CSC 1.5) Architecture

4.5.1 Setup (CSC 1.5.1)

The CSUs for this CSC are Identify Environment Data and Generate Scripts.

The design for Setup has not been modified since version 2.0.

4.5.1.1 Identify Environment Data (CSU 1.5.1.1)

The design for this CSU has not been modified since version 2.0. The only changes are those made to accommodate the new tests for Ada 95 and the use of default choices for users.

4.5.1.2 Generate Scripts (CSU 1.5.1.2)

The design for this CSU has not been modified since version 2.0. The only changes are those made to accommodate the use of default choices for users.

4.5.2 Scripts (CSC 1.5.2)

The CSUs for this CSC are Control Testing and Control Reporting. A Quick-Look script controls the entire Quick-Look process by invoking the Control Testing script and the Control Reporting script. The Control Testing portion of this script directs its output to the Test Log file, which serves as input to the Control Reporting portion.

4.5.2.1 Control Testing (CSU 1.5.2.1)

The design for Control Testing CSU has not been modified since version 2.0.

4.5.2.2 Control Reporting (CSU 1.5.2.2)

The design for Control Reporting CSU has not been modified since version 2.0.

4.5.3 Report (CSC 1.5.3)

The design for the Report CSC has not been modified since version 2.0. The only CSU for this CSC is Report which has not been changes since version 2.0 and therefore the design is not repeated in this document.

5.0 CSCI DATA

There is no new CSCI data for the ACES Version 2.1.

6.0 CSCI DATA FILES

This section identifies the data files associated with the Support Software CSCI 1.0.

6.1 CSCI Data Files

6.1.1 Single System Analysis

There is a new output report file for Single System Analysis.

Purpose: This data file is a report from the Single System Analysis which outputs the name of the comparison group and the ratio of the fastest to the slowest time measurement in each comparison. The comparison groups will be divided into tables by speed advantage. The information will be taken from the existing SSA Multiple Comparison Reports. Results of comparisons of the data from the fastest to the slowest test problems in each comparison group will determine in which table it shall be placed.

6.1.2 Comparative Analysis

There is a new output report file for Comparative Analysis.

Purpose: This data file is a report from Comparative Analysis which outputs a table of the execution time ratios for each group for each system. It will then output the number of tests of each group successfully completed by each system. It will take this information from the present System Factors for all groups for all systems and the Summary of Successes for each group reports.

6.1.3 Harness

There is a new input file to the Harness software.

Purpose: This data file will be read by the Harness software and will identify the performance issues and the performance tests that are related to each performance issue.

7.0 REQUIREMENTS TRACEABILITY

Table 7.0-1 identifies the requirements for updating the ACES from version 2.0 through 2.1. They are pulled from the Usability Enhancements Document and the Operational Software Objectives Document.

The requirement IDs were generated using the following technique. The first three characters indicate whether it concerns support software (SPT) or operational software (OPS). This is followed by a dash and a single letter which indicates if it refers to pretest (P), harness (H), quick-look (QL), analysis (A), performance test (T) or documentation (D).

The Requirement Text column is a narrative of the requirement. The Alloc. column identifies the CSCs that satisfy the requirement. The Alloc. column will contain "ACES", "DOC", or CSC identifier(s). When "ACES" is found, the requirement is a system level requirement and is not specifically allocated to software. When "DOC" is found, the requirement is allocated to documentation.

The IAD column identifies how the requirement is to be validated. An "I" indicates that the requirement is satisfied "in accordance with" another requirement. When an "I" is found it will be followed by a Requirement ID. An "I" means that this requirement is satisfied when the named requirement is satisfied. An "A" in this column indicates that the requirement will be satisfied through Analysis. The tester will analyze results, source code, etc. to ensure that the requirement has been satisfied. A "D" in this column indicates that the requirement will be satisfied through Demonstration. A test will be created that will demonstrate that the requirement has been satisfied.

The Fix/Design Change column indicates whether the requirement necessitates a Fix to the existing software or whether the requirement will cause a change in the design of the version 2.0 software. When a design change is indicated the reader will find a description of that change in this document or in the Software Design Document for the Operational Software.

Table 7.0-1 Requirements Traceability Matrix

Requirement ID Requirement Text Alloc. IAD Fix/Design Change

OPS-D-6202
The Quick-Look and Performance Testing sections of the Primer shall warn the user that the Tasking and Delay_Timing tests using pragma Priority may not behave properly for Ada 95 implementations not supporting the Real-Time Annex. DOC A N/A

OPS-D-7201
The Primer shall inform the user that certain Tasking tests contain references to the implementation-dependent package "Starlet" and use the implementation-dependent "AST_Entry". The Primer shall give guidance for adapting these tests to implementations other than VAX-Ada. These tests have names of the form "tk_in_int_*", where '*' ranges from 01 to 10. They appear in the files with names of the form "tk_in#_.inc", where '#' ranges from 02 to 11. DOC A N/A

OPS-T-5301
The ACES shall determine the performance characteristics of conversions between enumeration or numeric (integer, modular, float, fixed, decimal) types and String, using attributes and instantiations of the generic IO packages in Text_IO. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5302
The ACES shall determine the performance characteristics of the "Succ" and "Pred" attributes for real types. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5303
The ACES shall determine the performance of a subprogram call when the subprogram is passed to the calling subprogram via an access-to-subprogram parameter. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5304
The ACES shall determine the performance characteristics of decimal type operations (if supported). CSC 1.3, 1.4, & 2.1 D D

OPS-T-5305
The ACES shall determine the performance impact of referencing and assigning to an object designated by a subprogram's formal access parameter. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5306
The ACES shall determine the performance of assignment of controlled types. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5307
The ACES shall determine the performance of user-defined finalization when exiting the scope of a controlled object. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5308
The ACES shall determine the time required for a call on a protected operation in the simplest of cases. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5309
The ACES shall determine the performance characteristics of the requeue statement. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5310
The ACES shall determine the performance characteristics of the delay until statement. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5311
The ACES shall investigate the performance characteristics of asynchronous transfer of control in various scenarios. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5312
The ACES shall determine the performance characteristics of references to entities in child packages. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5313
The ACES shall determine the performance characteristics of the operations in package Ada.Exceptions. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5314
The ACES shall determine the performance characteristics of the address arithmetic operations declared in the package System.Storage_Elements. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5315
The ACES shall determine the performance characteristics of the conversion operations declared in the package System.Address_To_Access_Conversions. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5316
The ACES shall determine the performance characteristics of the storage management operations declared in the package System.Storage_Pools. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5317
The ACES shall determine the performance characteristics of passing data among units by using the stream operations given by attributes. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5318
The ACES shall determine the performance characteristics of the operations declared in package Strings.Fixed. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5319
The ACES shall determine the performance characteristics of the operations declared in package Strings.Bounded. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5320
The ACES shall determine the performance characteristics of the operations declared in package Strings.Unbounded. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5321
The ACES shall determine the performance characteristics of the operations declared in Ada.Numerics.Elementary_Functions and in an instantiation of Ada.Numerics.Generic_Elementary_Function s, using Float. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5322
The ACES shall determine the performance characteristics of the random value function declared in instantiations of Ada.Numerics.Discrete_Random. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5323
The ACES shall determine the performance characteristics of the shift and rotate operations provided by package Ada.Interfaces. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5324
The ACES shall determine the performance characteristics of calls to subprograms written in C, C++, Fortran, and COBOL, as supported by the specific implementation. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5325
The ACES shall determine the performance characteristics of the operations provided by package Ada.Interfaces.C. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5326
The ACES shall determine the performance characteristics of the operations provided by package Ada.Interfaces.C.Strings. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5327
The ACES shall determine the performance characteristics of the operations provided by package Ada.Interfaces.C.Pointers. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5328
The ACES shall determine the performance characteristics of the operations provided by package Ada.Interfaces.COBOL. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5329
The ACES shall determine the performance characteristics of the operations provided by package Ada.Interfaces.Fortran. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5401
The ACES shall determine the performance impact of using aliased objects as subprogram parameters. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5402
The ACES shall determine the performance effects of declaring objects of unconstrained types (the initial constraint being taken from the initialization expression). CSC 1.3, 1.4, & 2.1 D D

OPS-T-5403
The ACES shall determine whether a performance cost is incurred when referencing an extension component of a tagged type. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5404
The ACES shall determine whether a performance cost is incurred in referencing and updating arrays and records whose components are aliased. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5405
The ACES shall determine the relative performance of dispatching for various kinds of controlling operands. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5406
The ACES shall determine the performance assignments to an aliased object and evaluations of an aliased object. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5407
The ACES shall determine the performance impact of using general access types to reference data at different accessibility levels. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5408
The ACES shall determine the performance characteristics of evaluations of the Access attribute. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5409
The ACES shall determine the relative performance of elaboration of private type and non-private type declarations. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5410
The ACES shall determine the performance impact of using discriminants for tasks and for protected objects. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5411
The ACES shall determine the performance impact of using exception handlers in accept bodies. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5412
The ACES shall determine the performance characteristics of exception propagation from protected subprograms and entries of protected objects. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5413
The ACES shall determine the performance impact of the use of representation clauses that have been added to the language. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5414
The ACES shall determine the performance characteristics of evaluations of the Bit_Order attribute. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5415
The ACES shall determine the performance impact of using unchecked conversion to produce valid and invalid values of a type. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5416
The ACES shall determine the performance impact of using the Unchecked_Access attribute to create access values. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5417
The ACES shall determine the performance characteristics of evaluations of the Storage_Pool and Storage_Size attributes for access subtypes. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5418
The ACES shall determine the performance characteristics of the operations declared in package Ada.Characters.Handling. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5419
The ACES shall determine the performance characteristics of Wide_Character operations. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5420
The ACES shall determine the performance characteristics of Wide_String operations. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5421
The ACES shall determine the performance characteristics of conversions between enumeration or numeric (integer, modular, float, fixed, decimal) types and Wide_String, using attributes and instantiations of the generic IO packages in Wide_Text_IO. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5422
The ACES shall determine the performance characteristics of the operations declared in package Strings.Wide_Fixed. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5423
The ACES shall determine the performance characteristics of the operations declared in package Strings.Wide_Bounded. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5424
The ACES shall determine the performance characteristics of the operations declared in package Strings.Wide_Unbounded. CSC 1.3, 1.4, & 2.1 D D

OPS-T-5425
The ACES shall determine the performance characteristics of the operations declared in package Streams.Stream_IO. Compare to the performance of equivalent attributes after applying the Stream function to create a Stream_Access value from the file. CSC 1.3, 1.4, & 2.1 D D

OPS-T-6101
Package specifications that do not require bodies but have them nevertheless shall include "pragma Elaborate_Body". See the files "sy_cum??.ada" where ?? is a two-digit integer ranging from 12 to 22. CSC 2.1 D F

OPS-T-6201
The "dt_dp*" series of tests shall issue their calls to "put_test_name" and "delay_t" from within the body of a task with the highest priority. The expression giving the value of "actual-delay-greater-than-requested" shall use the ">=" operator. The value of this variable shall be set to False at the end of each test. CSC 2.1 D F

OPS-T-7101
The following tests shall be renamed:

a9_ms_atc_taken to tk_at_atc_taken_01
a9_ms_atc_not_taken to tk_at_atc_not_taken_02
a9_ms_formal_acc_subp_01 to gn_su_formal_acc_subp_01
a9_ms_formal_package_01 to gn_su_formal_package_01
a9_ms_modular_oper_01 to ar_mt_modular_oper_01
a9_ms_modular_oper_02 to ar_mt_modular_oper_02
a9_ms_random_01 to ar_rn_random_float_01 CSC 1.3, 1.4, & 2.1 D F

OPS-T-7102
The following tests shall be renamed:

a9_ob_class_wide_dynamic_01 to oo_dp_class_wide_dynamic_01
a9_ob_class_wide_static_01 to oo_dp_class_wide_static_01
a9_ob_parms_01_deriv to oo_tx_parms_deriv_01
a9_ob_parms_05_deriv to oo_tx_parms_deriv_05
a9_ob_parms_10_deriv to oo_tx_parms_deriv_10
a9_ob_ext_aggregate_01 to oo_tx_ext_aggregate_01
a9_ob_ext_aggregate_02 to oo_tx_ext_aggregate_02
a9_ob_simp_aggregate_01 to oo_tx_simp_aggregate_01
a9_ob_simp_aggregate_02 to oo_tx_simp_aggregate_02 CSC 1.3, 1.4, & 2.1 D F

OPS-T-7103
The following tests shall be renamed:

a9_pt_prot_access_01 to pt_po_prot_access_01
a9_pt_prot_access_02 to pt_po_prot_access_02
a9_pt_prot_access_03 to pt_po_prot_access_03 CSC 1.3, 1.4, & 2.1 D F

SPT-A-5201
The Analysis software shall recognize the names of new performance test groups, subgroups, tests, and default main programs. CSC 1.4 A F

SPT-A-5202
The Menu program shall allow selection of groups consisting entirely of Ada 95 tests if and only if at least one selected implementation has been identified as an Ada 95 implementation. CSC 1.4 D F

SPT-A-5203
The Comparative Analysis program shall incorporate results from new performance test groups, subgroups, default main programs, and tests into its analyses and reports. CSC 1.4 A F

SPT-A-5204
The Single-System Analysis program shall provide comparisons of sets of new tests and of sets containing both new and previously existing tests as appropriate. CSC 1.4 D F

SPT-A-6301
The Analysis software shall provide default reports which shall be designed to provide the user with the least amount of information which would also be usable. The first CA report shall present the System Factors for all test groups. The second shall present the number of tests successfully implemented. CSC 1.4 D D

SPT-A-6302
The default metric for Comparative Analysis reports shall be execution time. CSC 1.4 D D

SPT-A-6303
The Analysis software shall provide default reports which shall be designed to provide the user with the least amount of information which would also be usable. The SSA report shall present a ratio of the smallest to the largest reported Language Features for each test comparison. CSC 1.4 D D

SPT-A-7301
The ACES shall provide a method to perform Comparative Analysis on User-Defined benchmarks. CSC 1.4 D F

SPT-A-7302
The ACES shall provide a method to perform Single-System analysis on User-Defined benchmarks. CSC 1.4 D F

SPT-A-8301
The Menu software shall allow the user to advance to the next screen without having to type "n" in addition to the command where multiple selections before going to the next screen are not possible. CSC 1.4 D D

SPT-A-8302
The Analysis software, on the MAIN MENU screen, shall tell the user to select a command or a tool and a command (separating them with a comma) instead of telling the user to "select one tool separated by a comma." CSC 1.4 D F

SPT-A-8303
The Analysis software shall be designed to exit gracefully if multiple errors come up after all selections are made instead of having the user resort to a keyboard abortion command. CSC 1.4 D F

SPT-A-8304
The Analysis software shall be designed to give adequate error messages and exit gracefully if it is missing a file so that the user will know what files are required for it to function properly. The software shall indicate if the file is missing or if it cannot locate the name it was given to find and where to make that change. CSC 1.4 A F

SPT-A-8305
The Analysis software shall be redesigned so that the GROUPS menu either fits on one screen or appears on two screens in a way that is easy to use. Presently the title and "1. Applications" scroll off screen when Ada_95 is an option. CSC 1.4 D D

SPT-A-8306
The Analysis software shall be redesigned so that the RUN OR SAVE REQUEST menu will indicate in all cases either that no reports were chosen (and therefore no output is to be expected) or what has been chosen and what will be produced. CSC 1.4 D F

SPT-A-8307
The Analysis software shall be revised so that when a screen indicates that one of a group of items must be selected, then the user will really have to make a selection before going to the next screen.

This occurs on at least the following screens: - SUMMARY OF ALL GROUPS LEVEL REPORT OPTIONS: - SSA MAIN REPORT OPTIONS - SSA HLS REPORT OPTIONS CSC 1.4 D F

SPT-A-8308
The Analysis software shall present a new menu that is a combination of the present MAIN and INITIAL menus. CSC 1.4 D D

SPT-A-8311
The Analysis software menus shall use language consistent with the actual reports. This shall include but not be limited to: - Selection 7 of the SUMMARY OF ALL GROUPS LEVEL REPORT OPTIONS, which indicates on the screen "errors" and on the report "successes". CSC 1.4 D F

SPT-A-8322
The Analysis menus shall be revised as necessary to be more self explanatory. This shall include but not be limited to the use of white space to separate selection areas and the re-wording of selections to reflect the actual content of the output. DOC D F

SPT-D-6120
The Primer shall define default options to help the user who starts at download and ends with reports. The Primer shall make accommodations for those users who enter at an intermediate stage, referencing those files that are necessary at each point. DOC D D

SPT-D-6121
The Primer shall note that for Pretest, the user has the option of unzipping only the files needed to compile and run Setup, Harness and Analysis. Test files can be added later at the user's discretion if there is a problem with space. DOC D F

SPT-D-6125
The Pretest section of the Primer shall be redesigned so that each step has four parts: 1. PREPARATION: what needs to be done before executing the step 2. EXECUTION: (if all of the preparation is complete, user can now run the script) include comments such as expected time on a system 3. RESULTS: list of files --- vertical rather than horizontal. In the case of long lists, multiple columns can be used. 4. DEFAULT: a special section for those running in the default mode. DOC D D

SPT-D-6126
The Primer shall indicate that the user need only unzip the support.zip file, go to the "working" directory, make a library, compile and link the Setup files and run Setup. After that the automated steps can be executed. DOC D D

SPT-D-6128
The Quick-Look section of the Primer shall be redesigned so that each step has four parts: 1. PREPARATION: what needs to be done before executing the step 2. EXECUTION: (if all of the preparation is complete, user can now run the script) include comments such as expected time on a system 3. RESULTS: list of files --- vertical rather than horizontal. In the case of long lists, multiple columns can be used. 4. DEFAULT: a special section for those running in the default mode. DOC D D

SPT-D-6129
The Primer shall indicate that the user need only unzip the support.zip file, go to the "working" directory, make a library, compile and link the Quick-Look Setup file and run the Quick-Look version of Setup. After that the automated steps can be executed. DOC D D

SPT-D-6207
The Primer, Section 4, shall reflect the Harness modifications described elsewhere in this document. In particular, the algorithm shall be modified to add a description of the additional activities described in requirement ID's SPT-H-(6201-6206). DOC A N/A

SPT-D-6208
Section 4.1.1 of the Primer shall describe any additional prerequisites as a result of the requirements of requirement ID's SPT-H- (6201-6206). DOC A N/A

SPT-D-6209
The Primer shall contain one or more sections (probably numbered 4.1.2, with later sections renumbered) describing the predefined test selection process described in requirement ID's SPT-H-(6201-6206). DOC A N/A

SPT-D-6210
Section 6.1 of the User's Guide shall reflect the modifications to Harness described in requirement ID's SPT-H-(6201-6206). DOC A N/A

SPT-D-6211
Section 6.3 of the User's Guide shall be modified to reflect the modifications to Harness described in requirement ID's SPT- H-(6201-6206). DOC A N/A

SPT-D-6212
The User's Guide shall contain a new section (probably numbered 6.4, with later sections renumbered) describing the predefined test selection added to Harness. DOC A N/A

SPT-D-7401
The ACES documentation shall provide a detailed description of how benchmark tests are currently constructed. DOC D F

SPT-D-7402
The ACES documentation shall provide a detailed description of what a user must do in order to create a user defined benchmark. DOC D F

SPT-D-7403
The ACES documentation shall provide a detailed description of what the user must do in order to incorporate a user defined benchmark into the current ACES structure. DOC D F

SPT-D-7404
The ACES documentation shall provide a detailed description of what the user must do in order to capture the results generated by a user defined benchmark. DOC D F

SPT-D-7405
The ACES documentation shall provide a detailed description of what the user must do in order to perform Comparative Analysis using results from a user defined benchmark. DOC D F

SPT-D-7406
The ACES documentation shall provide a detailed description of what the user must do in order to perform Single System Analysis using results from a user defined benchmark. DOC D F

SPT-D-7407
The ACES documentation shall provide examples of the steps that the user must perform when incorporating user defined benchmarks. DOC D F

SPT-D-8101
The Primer shall indicate to the user at each step what files must be present before that step can be executed. This shall be indicated either directly or in the form of an exact reference to the required information. The user shall not be expected to be conversant with all of the documentation before reaching the point in question. This shall be consistent with each possible combination of choices and shall include but not be limited to Section 3.2.4.a which does not differentiate between what files need to be in the directory if CPU rather than EL is selected. DOC D D

SPT-D-8102
The Primer shall present a cross reference index for ACES Documentation that is verified and updated. DOC D F

SPT-D-8103
The Primer documentation shall list at the end of Section 3 all of the referenced files, their actual file names, and which steps use them. DOC D D

SPT-D-8104
The Primer shall properly and clearly document the corresponding software. This will eliminate existing errors and clarify existing documentation. These shall include but will not be limited to the following: - Section 2.1.2: Clarify references to Harness tests or testing Harness. - Section 2.2: Incorrect counts of software and file names when it refers to "Versions" of ACES and to seven compressed files. DOC D F

SPT-D-8105
The Primer shall define all Primer-specific notation. This shall include but is not limited to the last paragraph of Section 2 which gives no sample for # sign but uses it in Section 2.1.3 and other places. DOC D F

SPT-D-8106
The User's Guide shall properly and clearly document the corresponding software to eliminate existing errors and clarify existing documentation. These shall include but will not be limited to the following: - Section 5.1.4 (first bullet) which refers to Ada 95 MATCH packages in places rather than Ada 95 MATH packages. DOC D F

SPT-D-8108
The Primer shall indicate the room needed to house the uncompressed files as in Section 2.2, paragraph 4 ("The distributed...working directory.") and elsewhere as needed. DOC D F

SPT-D-8109
The Primer shall indicate the correct orientation of files. If they are unzipped according to instructions, an "aces" subdirectory will be created off the user's directory and the working, source, support, tests, assessrs, qlook, and docs subdirectories will be created off the newly created aces directory. The individual test subdirectories will be created off the tests directory and the individual assessor subdirectories will be created off the assessrs directory. This is incorrect in the present Figure 2-1 and in the text following that figure. DOC D F

SPT-D-8110
The Primer shall indicate at what point the Setup executable can actually be run. Sections 3.1, 3.1.1, and 3.1.2 say what to do before running Setup and what to do once Setup is run but the user can read on for several paragraphs before realizing that setup should have been executed. DOC D F

SPT-D-8111
The Primer shall clarify the directions for UNIX users. These shall include but not be limited to: - Section 3.1.3.2 paragraph 1: The copying and renaming are now done automatically; - Section 3.1.3.2 paragraph 3: Bourne shell or C shell; and - Section 3.1.3.2 paragraph 4: Need to mention that the example given will run in the back ground. DOC D F

SPT-D-8112
The Primer shall indicate instances in which there may be a difference in what the user can expect when running Ada 83 as opposed to running Ada 95. These shall include but not be limited to: - Section 3.2.11.c: If one is running an Ada 95 compiler, the files specific to Ada 95 will be present as well. DOC A F

SPT-D-8113
The Primer shall indicate which files will be automatically deleted in each step, and which can or should be manually deleted. This shall include but not be limited to Section 3.2.e. DOC D F

SPT-D-8114
The User's Guide shall present a clear and succinct description of each item covered. This shall include but not be limited to Section 5.1.1 paragraphs one and two which need to be rewritten and combined. The second paragraph looks as if it should be in the middle of the first. DOC D F

SPT-D-8115
The Primer shall make better and more consistent use of CAPITALIZATION, underlines, italics, and boldface. This shall include but not be limited to Section 3.2.5.a in which "source" should be in boldface. DOC D F

SPT-D-8118
The Quick-Look section of the Primer shall match the Pretest section where appropriate. Note Primer 8.2.2.2.d. DOC D F

SPT-D-8202
The User's Guide shall be careful of maintaining adequate references so that they are complete and accurate. This shall include but not be limited to: - The last paragraph of Section 5.4.3 which refers to Sections 9.1.6 and 6.13 of the User's Guide as if they are in the Reader's Guide and gives no indication as to where in the Reader's Guide one can find a discussion of the interpretation of such modified tests. DOC A N/A

SPT-D-8203
The Harness documentation shall list at the end of each section all default file names, their original locations, when they are needed, how they are referenced, when they are revised by a specific part of the program, and if the user changes the file name, where else changes need to be made to be consistent. DOC A N/A

SPT-D-8204
The User's Guide shall warn the user that if either the "test.sel" or the "test.sum" file is corrupted, Harness will exit gracefully but the condition can be cured by deleting the offending file and running Harness without it. Harness will happily build another. References are made in Section 6.2.5 and Section 6.2.6. DOC A N/A

SPT-D-8205
The User's Guide shall explain "status" in reference to Harness as referenced in Section 6.4.4, Section 6.4.5, Section 6.5.1 and elsewhere. If "status" is explained in detail elsewhere, then that section needs to be referenced at this point. DOC A N/A

SPT-D-8206
The User's Guide shall clarify whether the "current subgroup" can be in a different group than the "current group". This shall include but not be limited to: - Section 6.5.12 DOC A N/A

SPT-D-8207
The User's Guide shall demonstrate consistency between its text and its graphics. This shall include but not be limited to: - Section 6.5.2 which refers to "in the groups menu below, the classical group ... number (3)" however, the screen in Figure 6-4 shows groups 11-17. DOC A N/A

SPT-D-8208
The User's Guide shall fully explain each and every file it references. This shall include but not be limited to: - Section 6.9.3 which displays in Figure 6- 11 the result of the Get_Test_Name_List (GT) command and prompts the user for a "Test file name" but does not indicate what information in what format should be in that file and whether there is a default or sample file that the user can reference. DOC A N/A

SPT-D-8209
The User's Guide shall provide the user with adequate information on how to modify files. This shall include but not be limited to text based data files (e.g., zh_??.txt, za_cosys.txt, zh_??.lib) that the user is to modify but that expect a specific number of white space delimiters between what appear on the screen to be columns of figures. DOC A N/A

SPT-D-8210
The User's Guide shall present information about the ACES in a clear, concise, logical and factual manner. This clarification shall include but not be limited to: - Section 6.13. - The problem referred to in the last paragraph of Section 6.1 which mentions an "inconsistency in the Harness". DOC A N/A

SPT-D-8309
The Analysis software shall define the help menu for BUILD/MODIFY SYSTEM NAME FILE (GENERAL INFORMATION) so that the file names explained there are the same as those that appear in selections and make other corrections as needs so that the help menus correspond to the screens they serve. CSC 1.4 D F

SPT-D-8310
The Analysis documentation shall list at the end of each section all default file names, their original locations, when they are needed, how they are referenced, when they are revised by a specific part of the program, and if the user changes the file name, where else changes need to be made to be consistent. DOC D F

SPT-D-8312
The Primer shall give clear, concise, and logical explanations for ACES. This shall include but not be limited to the following section: - Section 3.2.12.c: "introduces this step" -- - for which paragraph and for what purpose? - Section 3.2.13.e.3: system1 is an Ada 83. What if they are testing an Ada 95? DOC D F

SPT-D-8313
The User's Guide shall provide thorough explanations of all of the possibilities to which it refers or provide adequate reference to resources in which the information can be easily found. This shall include but not be limited to: - Section 9.3 par 4 refers to za_anlys.com which cannot be found. - Figure 9-27 indicates that the View/Modify systems selection is number 5 when it is actually two more than the number of systems listed. DOC D F

SPT-D-8314
The User's Guide shall fully explain the importance of data files, how to manually construct them, what programs need which ones and will modify them, and where sample/dummy files can be found. This shall include but not be limited to: - Section 9.3.2 tells the user to construct the file za_cosys.txt and refers the reader to Section 9.1.4 but both sections fail to mention the fact that a dummy comes with the support.zip file and can be easily modified when running the Analysis MENU program. DOC A F

SPT-D-8315
The User's Guide shall fully explain how Condense, CA and SSA work both with the Menu and separately and what files are needed for either menu or independent use. This shall include but not be limited to clarification of several places in Section 9, including Section 9.1.1, which refers to Figure 9-1 and the fact that Condense, CA and SSA can be run independently of the Menu but give no more details as to where the files indicated in Figure 9-1 can be found. It also needs to cross reference the fact that Step 12 in the Primer will compile this automatically. It also refers to "za_anlys.com" which is not part of the test suite. Much of the information in Section 9 tends to ignore the fact that the Primer and the menu created in Step 12 of Pretest exist, as well as the fact that running the files created by Harness will produce log files that can be used by Analysis. DOC D F

SPT-D-8316
The Reader's Guide shall clarify its use of the phrase "classical benchmark tests" by making appropriate reference to the Classical (Cl_*) performance tests. It shall further refer to "benchmark test suite" as the "ACES performance tests". This shall include, but not be limited to: - the Abstract - Section 2.4.2.4: The paragraph "Represent classic benchmark ... available for other languages" - Section 3.2.8.1 - Section 5, paragraph 1 - Section 5.4.2, paragraph 2 DOC D F

SPT-D-8317
The Reader's Guide shall list at the end of each section all default file names, their original locations, when they are needed, how they are referenced, when they are revised by a specific part of the program, and if the user changes the file name, where else changes need to be made to be consistent. This shall include but not be limited to: - Section 5.2.2 in which no default names are given for the No Data, Exceptional Data, or Multiple Results Reports. DOC D F

SPT-D-8318
The Reader's Guide shall give a listing of all of the possible Analysis reports and the implications of the results along with cautions as to what could cause differences that would make the report misleading. This shall include clarification of but not be limited to: - Section 5.3.2.1.2 (and others) which refer to Figure X-x, name the report that is displayed and sometimes give a short explanation of what each shows. These should be expanded in their definitions and separated so that they follow the reports that they describe. DOC D F

SPT-D-8319
The Reader's Guide shall accurately reflect changes caused by new versions. This shall include but not be limited to: - The third paragraph (Gps:) of Section 5.3.2.2.3 which refers to the number of defined groups as 17. DOC D F

SPT-D-8320
The Reader's Guide shall explain in appropriate detail each of the possibilities to which it refers. This shall include but not be limited to: - The fifth paragraph of Section 5.3.2.2.3 (NoData:) which refers to the user entering the appropriate code but does not tell what codes are appropriate or where they can be entered or what will happen if they are or are not entered. DOC D F

SPT-D-8321
The User's Guide shall adequately explain all of the adaptations and limitations. This shall include but not be limited to Section 9.6 and especially the last limitation ("The Analysis... batch mode"). Since only 17 groups can now be displayed on a crt screen, what is meant by menu, (26) items, (25) systems, and groups. In addition, references to sections explaining requirements shall be instituted. DOC D F

SPT-D-8401
The Primer shall correctly, concisely and adequately explain the Quick-Look section of the ACES. This shall include but not be limited to: - the last sentence of the paragraph following Section 8.2.2.j which refers to "compilable Ada --". DOC D F

SPT-D-8501
The Reader's Guide shall present information in a clear, concise, logical, and factual manner. This shall include, but not be limited to: - Section 3.2.2: This section refers to pragmas in the first list, but does not indicate which list is the first one. - Section 3.2.2: All lists shall be vertical. If they are too long, multiple columns are acceptable. - Section 3.2.4.1.7: The statistics in paragraph 3 are incorrect. They should be the opposite, i.e., exchange the words inner and outer. - Section 3.2.7.3: "...until the I/O completes" transitive verb needs an object. DOC D F

SPT-H-5101
The Harness shall recognize new performance test groups and subgroups, as determined by new and renamed performance tests. CSC 1.3 D F

SPT-H-5102
The Harness shall recognize names of new default main programs that incorporate new and renamed tests. CSC 1.3 D F

SPT-H-5103
Main programs that incorporate new or renamed tests shall not include any previously existing tests. CSC 1.3 D F

SPT-H-5104
The Harness shall display all new test names that are appropriate for the implementation being tested. CSC 1.3 D F

SPT-H-5105
The Harness shall display tests specific to Ada 95 only if the user has identified the implementation as an Ada 95 implementation. CSC 1.3 D F

SPT-H-5106
The Harness shall display group and subgroup names only if the groups and subgroups contain tests applicable to the language version of the implementation under test. CSC 1.3 D F

SPT-H-6201
The Harness software shall provide the user an option of selecting tests based on the performance issues of interest to the user. CSC 1.3 D D

SPT-H-6202
The Harness shall provide the user with the option of bypassing the performance issues menu and proceeding directly to the Harness menu. CSC 1.3 D D

SPT-H-6203
The Harness shall provide a default set of performance issues, which shall be preselected when the performance issues menu first appears. CSC 1.3 D D

SPT-H-6204
The Harness shall allow the user to select or de-select any of the performance issues. CSC 1.3 D D

SPT-H-6205
The Harness shall provide at least the following options on the performance issues menu: - select all performance issue selections; - deselect all performance issue selections; - request help; and - quit the performance issues menu (exit to the Groups menu). CSC 1.3 D D

SPT-H-6206
If the performance issues menu is not used, the ACES shall work in the same manner as in ACES, Version 2.0. If a test.sel file exists, Harness shall use that as the initial test selection. Otherwise, no tests will be selected when the Harness menu is initially displayed. CSC 1.3 D D

SPT-H-7201
The Harness shall provide a mechanism for selecting User-Defined benchmarks in a fashion similar to the way that tests are currently handled in the ACES. CSC 1.3 D D

SPT-H-7202
The Harness shall provide a mechanism for generating command scripts that will execute User-Defined benchmarks in a fashion similar to the way that command scripts are currently generated by the Harness. CSC 1.3 D D

SPT-H-8201
The Harness software shall indicate, in its error messages in UPDATING, the name of the file causing the error rather than the name of the file following the file that caused the error. CSC 1.3 D F

SPT-P-6101
The Pretest software shall provide the user with the opportunity to exercise the same degree of detailed control of the pretest activity process that is provided by the Version 2.0 software. CSC 1.1 A F

SPT-P-6102
The number of questions presented to the user by the Pretest software shall be minimized by the use of default values and values that are specific to the operating system and compiler family being tested. CSC 1.1 A D

SPT-P-6103
The Pretest software shall base the default values for the following upon the operating system and compiler family being used. If no operating system or compiler family is identified, the user shall be required to supply the values.
* Command to execute a compiled program on the host system or on a self-hosted system;
* Command to delete a file;
* Suffix required for the delete command;
* Command to invoke a command script file;
* Command to copy a file;
* Command to rename a file;
* Symbol for the current directory;
* Symbol used as a separator in directory paths;
* Symbol used as the terminator of a directory path;
* Special first line of a script file (allow for multiple lines);
* Special symbol for the beginning of a script file line;
* Symbol or word to indicate the beginning of a comment line;
* Command to echo a string or value to the output file;
* Delimiter for a string being echoed to the output file;
* Suffix used to direct echoed string to a particular file;
* Symbol for the first parameter to a script file;
* Special form of the first parameter symbol for use in an echoed string;
* Symbol for the second parameter to a script file;
* Command to identify the Ada library for compiling to run on the host system (or a self-hosted system);
* Suffix expected for an Ada source code file;
* Command to compile a program (with default options) to be executed on the host system (or a self-hosted system);
* Command to compile (with full optimization) a program to be run on the target system (or a self-hosted system);
* Command to compile (with no optimization) a program to be run on the target system (or a self-hosted system);
* Command to compile (with full checking enabled) a program to be run on the target system (or a self-hosted system);
* Command to link a program to run on the target system (or a self-hosted system);
* Command to delete a compilation unit from the current Ada library;
* Command to delete the body (only) of a compilation unit from the current Ada library; and
* Command to delete the specification (only) of a compilation unit from the current Ada library. CSC 1.1 A D

SPT-P-6104
For commands and values that depend on the operating system and compiler family, the Pretest software shall make use of default responses applicable to the following combinations:

Operating System Compiler Family

DOS ThomSoft/Alsys DOS GNAT DOS Unspecified UNIX ThomSoft/Alsys UNIX Rational/VADS UNIX GNAT UNIX Unspecified VMS VAX-Ada VMS Unspecified Unspecified ThomSoft/Alsys Unspecified GNAT Unspecified Rational/VADS CSC 1.1 A D

SPT-P-6105
The Pretest software shall provide instructions or shall refer to documentation containing instructions for providing user-defined defaults for the commands and values that are specified for the above (SPT-P-6104) combinations of operating system and compiler family. CSC 1.1 A F

SPT-P-6106
The Pretest software shall ask the user to provide the following information in all cases:
* The operating system being used (for the host system);
* The compiler family being used;
* Whether the system is for cross- compilation or is self-hosted;
* Whether the system implements Ada 83 or Ada 95; and
* The directory path for the "aces" root directory. CSC 1.1 D D

SPT-P-6107
The Pretest software shall assume the indicated values for the following:
* Execution time measurement (EL: Elapsed time)
* Compilation time measurement (EL: Elapsed time)
* Code size measurement technique (L: Label'Address)
* Host or self-hosted executable suffix (.exe)
* Bit size of largest integer type (32)
* Default file name extension for script files will be: - DOS - .bat - UNIX - .unx - VMS - .com CSC 1.1 A D

SPT-P-6108
The Pretest software shall assume that the Portable math packages are to be used for the default setting if the system is an Ada 83 implementation; otherwise, it shall assume the existence of the math packages required by the Ada 95 standard. CSC 1.1 A D

SPT-P-6109
For cross-compilation systems only, the Pretest software shall require the user to provide the following commands and values:
* Command to define the Ada library for target system compilations;
* Command to compile a program to be run on the target system;
* Command to link a program to be run on the target system;
* Suffix for an executable file linked to run on the target system;
* Command to execute a compiled program on the target system;
* Command to delete the specification (only) of a unit from the target library;
* Command to delete the body (only) of a unit from the target library; and
* Command to delete both the specification and body of a unit from the target library. CSC 1.1 A D

SPT-P-6110
The Pretest software shall neither ask for nor use the following information that is requested by the Version 2.0 software:
* Command to execute a program under the debugger;
* Command to capture the status of a completed process;
* Command to export the status of a completing process;
* Command to compile with debugging enabled;
* Command to link a program for execution under the debugger;
* Method of managing the input and output files for the Include tool;
* Command for defining a logical symbol; and
* Operator used in defining a logical symbol. CSC 1.1 A D

SPT-P-6122
The support.zip file shall be revised to unload the initial files needed for Pretest into a "working" subdirectory of the "aces" directory. CSC 1.1 D D

SPT-P-6123
Pretest shall give the user the opportunity to modify the default as well as the previously constructed environment file by rerunning Setup. CSC 1.1 D D

SPT-P-6124
Setup shall be modified so that the Ada source files required for Pretest will be copied from the distribution subdirectories to the "source" subdirectory where they can be accessed for compilation. All other required files shall be copied from the distribution directories (note SPT-D-8109) to "working". This will be done at the beginning of the Pretest activity and all files will be renamed at that point if needed. CSC 1.1 D D

SPT-P-8107
Setup shall inform the user which (83/95) compiler version is selected in any existing environment file. It now only informs the user of the selection of cross/self compiler. CSC 1.1 D D

SPT-P-8116
Setup shall make a backup copy of any existing environment file before it offers the user a chance to make changes. It shall also offer the user the chance to revert to the old version by replacing quit with two choices: (1) generate new file with the new selections; or (2) exit and save none of the changes; go back to the old version. CSC 1.1 D D

SPT-P-8117
The Pretest software unit "zg_incld" (four versions) shall specify the type CHARACTER for at least one of the literals in the code "for i in 'a' .. 'z' loop". CSC 1.1 A D

SPT-Q-6111
The Quick-Look software shall provide the user with the opportunity to exercise the same degree of detailed control of the Quick-Look activity process that is provided by the Version 2.0 software. CSC 1.5 A F

SPT-Q-6112
The number of questions presented to the user by the Quick-Look software shall be minimized by the use of default values and values that are specific to the operating system and compiler family being tested. CSC 1.5 A D

SPT-Q-6113
The Quick-Look software shall base the default values for the following upon the operating system and compiler family being used. If no operating system or compiler family is identified, the user shall be required to supply the values.
* Command to execute a compiled program on the host system or on a self-hosted system;
* Command to delete a file;
* Suffix required for the delete command;
* Command to invoke a command script file;
* Command to copy a file;
* Command to rename a file;
* Symbol for the current directory;
* Symbol used as a separator in directory paths;
* Symbol used as the terminator of a directory path;
* Special first line of a script file (allow for multiple lines);
* Special symbol for the beginning of a script file line;
* Symbol or word to indicate the beginning of a comment line;
* Command to echo a string or value to the output file;
* Delimiter for a string being echoed to the output file;
* Suffix used to direct echoed string to a particular file;
* Symbol for the first parameter to a script file;
* Special form of the first parameter symbol for use in an echoed string;
* Symbol for the second parameter to a script file;
* Command to identify the Ada library for compiling to run on the host system (or a self-hosted system);
* Suffix expected for an Ada source code file;
* Command to compile a program (with default options) to be executed on the host system (or a self-hosted system);
* Command to compile (with full optimization) a program to be run on the target system (or a self-hosted system);
* Command to link a program to run on the target system (or a self-hosted system); and
* Command to delete a compilation unit from the current Ada library. CSC 1.5 A D

SPT-Q-6114
For commands and values that depend on the operating system and compiler family, the Quick-Look software shall make use of default responses applicable to the following combinations:

Operating System Compiler Family

DOS ThomSoft/Alsys
DOS GNAT
DOS Unspecified
UNIX ThomSoft/Alsys
UNIX Rational/VADS
UNIX GNAT
UNIX Unspecified
VMS VAX-Ada
VMS Unspecified
Unspecified ThomSoft/Alsys
Unspecified GNAT
Unspecified Rational/VADS
CSC 1.5
A D

SPT-Q-6115
The Quick-Look software shall provide instructions or shall refer to documentation containing instructions for providing user- defined defaults for the commands and values that are specified for the above combinations of operating system and compiler family. CSC 1.5 A F

SPT-Q-6116
The Quick-Look software shall ask the user to provide the following information in all cases:
* The operating system being used (for the host system);
* The compiler family being used;
* Whether the system is for cross- compilation or is self-hosted;
* Whether the system implements Ada 83 or Ada 95; and
* The directory path for the "aces" root directory. CSC 1.5 D D

SPT-Q-6117
The Quick-Look software shall assume the indicated values for the following:
* Host or self-hosted executable suffix (.exe)
* Bit size of largest integer type (32)
* Default file name extension for script files will be: - DOS - .bat - UNIX - .unx - VMS - .com CSC 1.5 A D

SPT-Q-6118
For cross-compilation systems only, the Quick-Look software shall require the user to provide the following commands and values:
* Command to define the Ada library for target system compilations;
* Command to compile a program to be run on the target system;
* Command to link a program to be run on the target system;
* Suffix for an executable file linked to run on the target system;
* Command to execute a compiled program on the target system;
* Command to delete the specification (only) of a unit from the target library;
* Command to delete the body (only) of a unit from the target library; and
* Command to delete both the specification and body of a unit from the target library. CSC 1.5 A D

SPT-Q-6119
The Quick-Look software shall neither ask for nor use the following information that is requested by the Version 2.0 software:
* Method of managing the input and output files for the Include tool;
* Command for defining a logical symbol; and
* Operator used in defining a logical symbol. CSC 1.5 A D

SPT-Q-6127
Setup shall be modified so that the Ada source files required for Quick-Look will be copied from the distribution subdirectories to the "source" subdirectory where they can be accessed for compilation. All other required files shall be copied from the distribution directories (note SPT-D-8109) to "ql_work". This will be done at the beginning of the Quick-Look activity and all files will be renamed at that point if needed. CSC 1.5 D D

SPT-Q-6130
The support.zip file shall be revised to unload the initial files needed for Quick-Look into a "ql_work" subdirectory of the "aces" directory. CSC 1.5 D D

SPT-Q-7101
The Quick-Look facility shall be flexible enough to allow additional User-Defined benchmark tests to be added. CSC 1.5 A D

SPT-Q-7102
The Quick-Look facility shall be able to report on the results of additional User- Defined benchmark tests. CSC 1.5

SPT-Q-8402
The Quick-Look software unit "zg_incld" shall specify the type CHARACTER for at least one of the literals in the code "for i in 'a' .. 'z' loop". CSC 1.5 A D





Table 7.0-2 Requirements Sources

Document Code Document Title SPT Usability Enhancements Document for the ACES Version 2.1 OPS Operational Software Objectives Document for the ACES Version 2.1

8.0 NOTES

8.1 Abbreviations, Acronyms

ACEC...Ada Compiler Evaluation Capability
ACES...Ada Compiler Evaluation System
CA...Comparative Analysis
CSC...Computer Software Component
CSCI...Computer Software Configuration Item
CSU...Compiler Software Unit

N/A...Not Applicable
PDL...Program Design Language
SDD...Software Design Document
SSA...Single System Analysis
SPS...Software Product Specification