Ada Compiler Evaluation System
Version Description Document
for
Version 2.1
FINAL
Contract Number F33600-92-D-0125
CDRL A0036
Prepared for:
High Order Language Control Facility
Business Software Support Branch
88 CG/SCTL
Wright-Patterson AFB OH 45433-5707
Prepared by:
CTA INCORPORATED
5100 Springfield Pike, Suite 100
Dayton OH 45431
This document identifies and describes Version 2.1 of the Ada Compiler Evaluation System (ACES). The Version Description Document (VDD) records data pertinent to the status and usage of the ACES Software Product.
APPENDICES
LIST OF TABLES
Table 1-2 Appendices Description 1-2
This Version Description Document (VDD) describes Version 2.1 of the Software Product of the Ada Compiler Evaluation System (ACES) System.
The purpose of the ACES is to provide a capability for quantitative evaluation of Ada compilation systems. The ACES system is a set of software test programs and associated support tools and procedures which will determine the performance characteristics of Ada compilation systems. This includes the capability to automatically compare the results obtained on different Ada compilation systems. Such comparisons will isolate language constructions where one optimization has particular problems relative to other compilers tested. The ACES will test for the presence of particular compiler optimization.
ACES software is comprised of the test suite and the support tools. For a list and description of the tests contained in the test suite, see Appendix A, "Test Problem Descriptions". The support tools consist of:
1. Setup - A tool which performs automatic adaptation of the pretest scripts.
2. Include - A tool to perform text inclusion into an Ada source text file. It will assist in adapting programs to particular targets.
3. Harness - A program that provides an interactive interface for selecting performance tests, tracking their status, and generating command files which will compile, link, and run the selected tests.
4. Analysis Menu - This program provides an interactive interface for calling any of the analysis programs.
5. Condense - A tool to extract the timing data, the code expansion data, compile speed data and certain ancillary data from the output produced by compiling and executing the Operational Software and to write this information in a format usable by the Comparative Analysis program and by the Single System Analysis program.
6. Comparative Analysis (CA) - A tool to statistically compare results of the performance tests of various systems generated by compiling and executing the Operational Software.
7. Single System Analysis (SSA) - A tool to analyze the performance results from a single system.
8. Quick-Look - A tool to set up and execute a predefined set of performance tests that report execution time only.
This document describes the ACES Software Product and the differences between this version (ACES 2.0) and the previous (ACES 1.1). It also describes the compilation units, programs, test programs, and sample data distributed with this release. This document contains several appendices with release-dependent information, making the Primer, Reader's Guide and the User's Guide less sensitive to new releases. See the following table for a brief description of each appendix included in this document.
Table 1-2 Appendices Description
APPENDIX CONTENTS A) Test Problem Descriptions List of test problem names with a brief description of each (New or withdrawn tests are identified) B) Test Problem to Source File Map List of test problems and the source file in which they are contained C) Distribution Description List of files in the distribution software D) System-Dependent Test Problems List of test problems which exercise system dependent features E) Debugger Assessor Description of the Debugger scenarios F) Diagnostic Assessor Description of the Diagnostic scenarios G) Capacity Assessor Description of the Capacity scenarios H) Library Assessor Description of the Library scenarios I) New and Modified Tests List of new performance tests for this release and a list of modified tests from ACES Version 1.0
The following documents are referenced in this VDD.
ANSI/MIL-STD-1815A Reference Manual for the Ada Programming Language
ISO/IEC 8652 (1995) Programming Language Ada, Language and Standard Libraries (RM 95)
User's Guide for Ada Compiler Evaluation System (ACES), Version 2.1
High Order Language Control Facility
Business Software Support Branch
88 CG/SCTL
Wright-Patterson AFB OH
Readers Guide for Ada Compiler Evaluation System (ACES), Version 2.1
High Order Language Control Facility
Business Software Support Branch
88 CG/SCTL
Wright-Patterson AFB OH
Primer for Ada Compiler Evaluation System (ACES), Version 2.1
High Order Language Control Facility
Business Software Support Branch
88 CG/SCTL
Wright-Patterson AFB OH
This section contains information on the inventory of materials released, the inventory of CSCI contents, the adaptation data, interface compatibility, bibliography of reference documents, installation instructions, and possible problems and known errors.
The ACES software and documentation are available by anonymous FTP from the host "sw-eng.falls-church.va.us" in the directory "public/AdaIC/testing/aces/v2.0" and from other Ada-related hosts. Document files are included in PostScript format and as ASCII text. These ACES files are also accessible via the World-Wide Web. The home page URL is "http://sw-eng.falls-church.va.us/AdaIC/testing/aces/".
For further information about the ACES, contact the High Order Language Control Facility. As of 1 March 1995, the appropriate contact is:
Mr. Brian Andrews
88 CG/SCTL
3810 Communications, Suite 1
Wright-Patterson AFB, OH 45433-5707
(513) 255-4472
The release of the Software Product of the ACES is comprised of:
* The distribution software
* The Primer
* The User's Guide
* The Reader's Guide
* This VDD
The ACES Software Product consists of two CSCIs: the Operational Software (test suite) and the Support Software (support tools). The distribution software contains the test suite and the support tools. For a list of the contents of the test suite, see Appendix A, "Test Problem Descriptions." The eight support tools are:
* Setup
* Include
* Harness
* Analysis Menu
* Condense
* Comparative Analysis (CA)
* Single System Analysis (SSA)
* Quick-Look
This paragraph has been tailored out.
This paragraph has been tailored out.
The ACES Software Product has no "unique-to-site" data. Appendix D, "System-Dependent Test Problems", identifies all system dependencies contained in the items being released.
Not applicable. For information on how the test suite and the support tools interface, refer to the User's Guide.
Refer to Section 2, "Referenced Documents".
This paragraph has been tailored out.
For information detailing how to install and check out the delivered ACES Software Product, refer to the Primer and/or User's Guide.
There are no known errors in this release of ACES.
4. NOTES
This list of abbreviations and acronyms is used throughout the entire ACES document set.
ACEC Ada Compiler Evaluation Capability
ACES Ada Compiler Evaluation System
ACM Association for Computing Machinery
ACM/SIGAda Association for Computing Machinery/Special Interest Group on Ada ACVC Ada Compiler Validation Capability
ACVC Ada Compiler Validation Capability
AES Ada Evaluation System (UK Ministry of Defence)
AJPO Ada Joint Program Office
CA Comparative Analysis
CISC Complex Instruction Set Computer
CPU Central Processing Unit
CRC Cyclic Redundancy Check
CSCI Computer Software Configuration Item
DEC Digital Equipment Corporation
DoD Department of Defense
HOLCF High Order Language Control Facility
I/O Input/Output
LRM (Ada) Language Reference Manual (ANSI/MIL-STD-1815A)
MCCR Mission Critical Computer Resource
MIP Million Instructions Per second (a measure of hardware speed)
MIS Management Information System
MoD Ministry of Defence
NUMWG Numerics Working Group (ACM SIGAda organization)
OS Operating System
PIWG Performance Issues Working Group
RAM Random Access Memory
RISC Reduced Instruction Set Computer
RM (Ada 95) Reference Manual (ISO/IEC 8652 (1995))
ROM Read Only Memory
RTS Run-Time System
SIGAda Special Interest Group on Ada (ACM sponsored organization)
SSA Single System Analysis (ACES analysis tool)
VAX Virtual Address eXtension (DEC family of processors)
VDD Version Description Document
VLIW Very Long Instruction Word
VMS Virtual Memory System (DEC operating system for VAX processors)