Ada Compiler Evaluation System

Version Description Document

for

Version 2.1

FINAL

Contract Number F33600-92-D-0125

CDRL A0036

Prepared for:

High Order Language Control Facility

Business Software Support Branch

88 CG/SCTL

Wright-Patterson AFB OH 45433-5707

Prepared by:

CTA INCORPORATED

5100 Springfield Pike, Suite 100

Dayton OH 45431

ABSTRACT

This document identifies and describes Version 2.1 of the Ada Compiler Evaluation System (ACES). The Version Description Document (VDD) records data pertinent to the status and usage of the ACES Software Product.

TABLE OF CONTENTS

APPENDICES

  • APPENDIX A - Test Problem Description
  • APPENDIX B - Test Problem to Source File Map
  • APPENDIX C - Distribution Description
  • APPENDIX D - System-Dependent Test Problems
  • APPENDIX E - Debugger Assessor
  • APPENDIX F - Diagnostic Assessor
  • APPENDIX G - Capacity Assessor
  • APPENDIX H - Library Assessor
  • APPENDIX I - New and Modified Tests

    LIST OF TABLES

    Table 1-2 Appendices Description 1-2

    1. SCOPE

    1.1 Identification

    This Version Description Document (VDD) describes Version 2.1 of the Software Product of the Ada Compiler Evaluation System (ACES) System.

    1.2 System Overview

    The purpose of the ACES is to provide a capability for quantitative evaluation of Ada compilation systems. The ACES system is a set of software test programs and associated support tools and procedures which will determine the performance characteristics of Ada compilation systems. This includes the capability to automatically compare the results obtained on different Ada compilation systems. Such comparisons will isolate language constructions where one optimization has particular problems relative to other compilers tested. The ACES will test for the presence of particular compiler optimization.

    ACES software is comprised of the test suite and the support tools. For a list and description of the tests contained in the test suite, see Appendix A, "Test Problem Descriptions". The support tools consist of:

    1. Setup - A tool which performs automatic adaptation of the pretest scripts.

    2. Include - A tool to perform text inclusion into an Ada source text file. It will assist in adapting programs to particular targets.

    3. Harness - A program that provides an interactive interface for selecting performance tests, tracking their status, and generating command files which will compile, link, and run the selected tests.

    4. Analysis Menu - This program provides an interactive interface for calling any of the analysis programs.

    5. Condense - A tool to extract the timing data, the code expansion data, compile speed data and certain ancillary data from the output produced by compiling and executing the Operational Software and to write this information in a format usable by the Comparative Analysis program and by the Single System Analysis program.

    6. Comparative Analysis (CA) - A tool to statistically compare results of the performance tests of various systems generated by compiling and executing the Operational Software.

    7. Single System Analysis (SSA) - A tool to analyze the performance results from a single system.

    8. Quick-Look - A tool to set up and execute a predefined set of performance tests that report execution time only.

    1.3 Document Overview

    This document describes the ACES Software Product and the differences between this version (ACES 2.0) and the previous (ACES 1.1). It also describes the compilation units, programs, test programs, and sample data distributed with this release. This document contains several appendices with release-dependent information, making the Primer, Reader's Guide and the User's Guide less sensitive to new releases. See the following table for a brief description of each appendix included in this document.

    Table 1-2 Appendices Description


                                                                              
                  APPENDIX                             CONTENTS               
                                                                              
    
    A) Test Problem Descriptions         List of test problem names with a    
                                         brief description of each            
                                         (New or withdrawn tests are          
                                         identified)                          
    
    B) Test Problem to Source File Map   List of test problems and the        
                                         source file in which they are        
                                         contained                            
    
    C) Distribution Description          List of files in the distribution    
                                         software                             
    
    D) System-Dependent Test Problems    List of test problems which          
                                         exercise system dependent features   
    
    E) Debugger Assessor                 Description of the Debugger          
                                         scenarios                            
    
    F) Diagnostic Assessor               Description of the Diagnostic        
                                         scenarios                            
    
    G) Capacity Assessor                 Description of the Capacity          
                                         scenarios                            
    
    H) Library Assessor                  Description of the Library           
                                         scenarios                            
    
    I) New and Modified Tests            List of new performance tests for    
                                         this release and a list of modified  
                                         tests from ACES Version 1.0          
    
    
    

    2. REFERENCED DOCUMENTS

    The following documents are referenced in this VDD.

    2.1 Government Documents

    ANSI/MIL-STD-1815A Reference Manual for the Ada Programming Language

    ISO/IEC 8652 (1995) Programming Language Ada, Language and Standard Libraries (RM 95)

    2.2 Non-Government Documents

    User's Guide for Ada Compiler Evaluation System (ACES), Version 2.1

    High Order Language Control Facility

    Business Software Support Branch

    88 CG/SCTL

    Wright-Patterson AFB OH

    Readers Guide for Ada Compiler Evaluation System (ACES), Version 2.1

    High Order Language Control Facility

    Business Software Support Branch

    88 CG/SCTL

    Wright-Patterson AFB OH

    Primer for Ada Compiler Evaluation System (ACES), Version 2.1

    High Order Language Control Facility

    Business Software Support Branch

    88 CG/SCTL

    Wright-Patterson AFB OH

    3. VERSION DESCRIPTION

    This section contains information on the inventory of materials released, the inventory of CSCI contents, the adaptation data, interface compatibility, bibliography of reference documents, installation instructions, and possible problems and known errors.

    The ACES software and documentation are available by anonymous FTP from the host "sw-eng.falls-church.va.us" in the directory "public/AdaIC/testing/aces/v2.0" and from other Ada-related hosts. Document files are included in PostScript format and as ASCII text. These ACES files are also accessible via the World-Wide Web. The home page URL is "http://sw-eng.falls-church.va.us/AdaIC/testing/aces/".

    For further information about the ACES, contact the High Order Language Control Facility. As of 1 March 1995, the appropriate contact is:

    Mr. Brian Andrews

    88 CG/SCTL

    3810 Communications, Suite 1

    Wright-Patterson AFB, OH 45433-5707

    (513) 255-4472

    3.1 Inventory Of Materials Released

    The release of the Software Product of the ACES is comprised of:

    * The distribution software

    * The Primer

    * The User's Guide

    * The Reader's Guide

    * This VDD

    3.2 Inventory Of CSCI Contents

    The ACES Software Product consists of two CSCIs: the Operational Software (test suite) and the Support Software (support tools). The distribution software contains the test suite and the support tools. For a list of the contents of the test suite, see Appendix A, "Test Problem Descriptions." The eight support tools are:

    * Setup

    * Include

    * Harness

    * Analysis Menu

    * Condense

    * Comparative Analysis (CA)

    * Single System Analysis (SSA)

    * Quick-Look

    3.3 Class I Changes Installed

    This paragraph has been tailored out.

    3.4 Class II Changes Installed

    This paragraph has been tailored out.

    3.5 Adaptation Data

    The ACES Software Product has no "unique-to-site" data. Appendix D, "System-Dependent Test Problems", identifies all system dependencies contained in the items being released.

    3.6 Interface Compatibility

    Not applicable. For information on how the test suite and the support tools interface, refer to the User's Guide.

    3.7 Bibliography Of Reference Documents

    Refer to Section 2, "Referenced Documents".

    3.8 Summary Of Change

    This paragraph has been tailored out.

    3.9 Installation Instructions

    For information detailing how to install and check out the delivered ACES Software Product, refer to the Primer and/or User's Guide.

    3.10 Possible Problems And Known Errors

    There are no known errors in this release of ACES.

    4. NOTES

    4.1 Abbreviations, Acronyms

    This list of abbreviations and acronyms is used throughout the entire ACES document set.

    ACEC Ada Compiler Evaluation Capability

    ACES Ada Compiler Evaluation System

    ACM Association for Computing Machinery

    ACM/SIGAda Association for Computing Machinery/Special Interest Group on Ada ACVC Ada Compiler Validation Capability

    ACVC Ada Compiler Validation Capability

    AES Ada Evaluation System (UK Ministry of Defence)

    AJPO Ada Joint Program Office

    CA Comparative Analysis

    CISC Complex Instruction Set Computer

    CPU Central Processing Unit

    CRC Cyclic Redundancy Check

    CSCI Computer Software Configuration Item

    DEC Digital Equipment Corporation

    DoD Department of Defense

    HOLCF High Order Language Control Facility

    I/O Input/Output

    LRM (Ada) Language Reference Manual (ANSI/MIL-STD-1815A)

    MCCR Mission Critical Computer Resource

    MIP Million Instructions Per second (a measure of hardware speed)

    MIS Management Information System

    MoD Ministry of Defence

    NUMWG Numerics Working Group (ACM SIGAda organization)

    OS Operating System

    PIWG Performance Issues Working Group

    RAM Random Access Memory

    RISC Reduced Instruction Set Computer

    RM (Ada 95) Reference Manual (ISO/IEC 8652 (1995))

    ROM Read Only Memory

    RTS Run-Time System

    SIGAda Special Interest Group on Ada (ACM sponsored organization)

    SSA Single System Analysis (ACES analysis tool)

    VAX Virtual Address eXtension (DEC family of processors)

    VDD Version Description Document

    VLIW Very Long Instruction Word

    VMS Virtual Memory System (DEC operating system for VAX processors)