SYMPTOM                                PROBLEM                             SOLUTION               
                                                                                                                 
                                                                                                                 

PRETEST                                                                                                          

                                                                                                                 

PRETEST: STEP 1                                                                                                  

                                                                                                                 

1. Error message: error opening        The Ada library was not created       Create the Ada library again,       
library for compilation.               properly or doesn't exist.            making sure "xc_setli.com" is       
                                                                             correct.                            

                                                                                                                 

2. Error message: directory not        The logical aces_working_directory    Reset the aces_working_directory    
found.                                 is not set properly.                  to the desired directory.           

                                                                                                                 

3. Error message: not a valid library  Adaptation of "zc_setli.com" is in    Redefine the adalib logical in      
directory.                             error.                                "zc_setli.com" and check the login  
                                                                             file.                               

                                                                                                                 

PRETEST: STEP 2                                                                                                  

                                                                                                                 

1. Exception while running the         16 bit architecture expects a two's   See the User's Guide Section 5.1.1  
subtract_address function in           complement for subtract function.     "Measuring Code Size (Pretest       
"zp_label.ada".                                                              Steps 0, 2)".                       

                                                                                                                 

PRETEST: STEP 3                                                                                                  

                                                                                                                 

1. Messages from zp_tcall/zp_tcal2     System clock test program is in       Check installation.                 
are not displayed every 60 seconds.    error.                                                                    

                                                                                                                 

PRETEST: STEP 5                                                                                                  

                                                                                                                 

1. "zp_adasp.com" cannot be adapted.   Compiler doesn't have an optimize     Skip the compile of "zg_init.*"     
                                       for space option.                     with that option in zg_base4.       


               SYMPTOM                                PROBLEM                             SOLUTION               
                                                                                                                 

PRETEST: STEP 5 (continued)                                                                                      

                                                                                                                 

2. Error message when compiling        System's integer is 16 bits for       In zg_glob5, change                 
"zg_glob5.ada", "base type of type     predefined integer types.             INTERGER32_TYPE to be a new         
INTEGER32 is too small - modify in                                           LONG_INTEGER.                       
zg_glob5".                                                                                                       

                                                                                                                 

3. "zm_genma.ada" fails to compile.    System will not accept large generic  Create non-generic version. See     
                                       units.                                User's Guide Section 5.1.6.1.4:     
                                                                             Making a Non-Generic Math           
                                                                             Package".                           

                                                                                                                 

4. Warning message output by ZG_INIT   Compilation system optimizes a null   Add call on an assembler routine    
or ZG_INLNE that LOOP_TIME was         statement embedded in the timing      to the body of procedure            
computed as zero (or near zero).       loop into a noop.  This makes the     ZG_GLOB3.PROC_SPOIL.  This should   
                                       measured times for all non-null test  thwart the optimizer.               
                                       problems too large (because in these                                      
                                       cases, the timing loop overhead is                                        
                                       present and will not be subtracted                                        
                                       off).                                                                     

                                                                                                                 

5. The value of LOOP_TIME determined   The compiler may have optimized the   Examine  generated  machine  code   
by ZG_INIT (or initialization section  timing loop (but not to zero).        for the  target to  see  if  all    
of ZG_INLNE) seems too fast to                                               Ada statements will  be executed    
execute the typical number of                                                on each iteration  of  the inner    
instructions on the target processor.                                        timing loop.  If it does, this      
                                                                             condition is not a problem  (the    
                                                                             target processor is simply faster   
                                                                             or more efficient that initially    
                                                                             expected);  if  it does  not,  add  
                                                                              call  on  an  assembler routine    
                                                                             to ZG_GLOB3.PROC_SPOIL.             

                                                                                                                 

               SYMPTOM                                PROBLEM                             SOLUTION               
                                                                                                                 

PERFORMANCE TESTS                                                                                                

                                                                                                                 

1. Timing loop produces many           Contention in system or noise in      Rerun program.                      
unreliable times and/or null times     clock measurements.                                                       
for problems which take zero space.                                                                              

                                                                                                                 

                                                                             Increase value of 2 variables:      
                                                                             BASIC_ITERATION_COUNT               
                                                                             (_UPPER and _LOWER).                

                                                                                                                 

                                                                                                                 

                                                                                                                 

2. Repeatable zero time measurements   A pipelined target processor may be   On some targets it is possible to   
for some small (non-zero) test         able to overlap execution of problem  add instructions without            
problems.                              code with loop overhead code,         increasing execution time; ACES     
                                       fitting it into "stall slots" which   performance test measurements       
                                       were unused in the null loop.         represent an incremental cost of a  
                                                                             problem relative to the null loop.  

                                                                                                                 

                                       Null loop times may be invalid        Rerun "zg_init.*" programs.         
                                       because of contention when                                                
                                       "zg_init.*" executed.                                                     

                                                                                                                 

                                       Compiler may optimize loop overhead   Examine machine code generated for  
                                       in the performance test but not null  loop overhead in null loop          
                                       loop.                                 ("zg_init.*") and in test problems  
                                                                             displaying anomaly.                 

                                                                                                                 

                                                                                                                 


               SYMPTOM                                PROBLEM                             SOLUTION               
                                                                                                                 

PERFORMANCE TESTS (continued)                                                                                    

                                                                                                                 

3. Library units are not deleted or    Probable implementation               See discussion in Section 5.4.2,    
there are problems adapting the        dependencies.                         "Delete Library Units", of User's   
zc_del* files.                                                               Guide.                              

                                                                                                                 

4. Many unreliable performance         System daemons, noise in clock.       Increase the value of 2 variables:  
measurements which don't disappear                                           BASIC_ITERATION_COUNT               
with re-running.                                                             (_UPPER and _LOWER).                

                                                                                                                 

                                                                             Consider switching from elapsed     
                                                                             time to CPU time on embedded        
                                                                             targets.                            

                                                                                                                 

                                                                             Increase error tolerances.          

                                                                                                                 

                                       Inherent variability on target NULL   Use zeros, check code expansion     
                                       or near-NULL problems which should    sizes.                              
                                       have execution time of zero.                                              

                                                                                                                 

                                       Some language features may be         No reliable performance             
                                       implemented in an inherently          measurement can be made in such     
                                       variable way on targets; e.g.,        cases.                              
                                       problems using heap management may                                        
                                       not reset system.                                                         

                                                                                                                 

                                       Test problem is faster than the       Use measurement options ".ELL" or   
                                       precision that the timing loop can    ".CPL" rather than ".ELG" or        
                                       reliably measure - see the zg_init    ".CPG"; that will allow reliable    
                                       output (in the "zg_cpy.*" files) for  measurements to be made on faster   
                                       information on precision.             problems.                           

               SYMPTOM                                PROBLEM                             SOLUTION               
                                                                                                                 

PERFORMANCE TESTS (continued)                                                                                    

                                                                                                                 

5. Warning messages on compiling or    System doesn't support PRAGMA         Modify the listed problem with the  
comment in documentation.              CONTROLLED.                           error code                          
                                                                             ERR_AT_COMPILATION_TIME.            

                                                                                                                 

                                       System doesn't support PRAGMA PACK.   Over   a   hundred   test           
                                                                             problems  use this  feature.  Many  
                                                                              involving  Boolean arrays and      
                                                                             UNCHECKED_CONVERSION will simply    
                                                                             not work as intended. Mark these    
                                                                             problems with error code            
                                                                             ERR_AT_COMPILATION_TIME.            

                                                                                                                 

                                       System doesn't support PRAGMA         Review system documentation - it    
                                       SHARED.                               may be that all variables  are      
                                                                             treated  as SHARED, in  which       
                                                                             case there  is  no problem.    If   
                                                                             this  is  not  the case, these      
                                                                             problems  should  be  marked with   
                                                                             error code                          
                                                                             ERR_AT_COMPILATION_TIME.            

                                                                                                                 

                                       System doesn't support PRAGMA         No problem.  Performance of         
                                       IN-LINE totally.                      non-supporting system will simply   
                                                                             be slower than for other systems.   

                                                                                                                 

                                                                                                                 

                                                                                                                 


               SYMPTOM                                PROBLEM                             SOLUTION               
                                                                                                                 

PERFORMANCE TESTS (continued)                                                                                    

                                                                                                                 

                                       System doesn't support preemptive     Review system documentation for     
                                       priority scheduling.                  settings for compilation options    
                                                                             or initialization calls which will  
                                                                             provide for full preemptive         
                                                                             priority scheduling.  Compilation   
                                                                             system must treat expiration of a   
                                                                             DELAY in high priority task as      
                                                                             sufficient to task-switch from low  
                                                                             priority task.                      

                                                                                                                 

6. Errors at compile time or           System doesn't support file I/O.      Skip the test problems and mark     
exceptions at run time.                                                      with the code ERR_DEPENDENT_TEST.   

                                                                                                                 

7. Program or system crashes.          Implementation-dependent problem.     Examine diagnostic  messages  and   
                                                                             the test  program.  Some  systems   
                                                                             impose unanticipated restrictions   
                                                                             which might not  be  listed  in     
                                                                             the  quarantined  test list.        
                                                                             Either  adapt  problem  or  mark    
                                                                             error code with                     
                                                                             ERR_DEPENDENT_TEST.                 

                                                                                                                 

                                       Error in compilation system.          Recompile problem using NOOPTIMIZE  
                                                                             compiler option.                    

                                                                                                                 

                                                                             Use system debugger to isolate      
                                                                             problem.  Use DB compilation        
                                                                             option and omit INCLUDE step to     
                                                                             simplify.                           

                                                                                                                 


               SYMPTOM                                PROBLEM                             SOLUTION               
                                                                                                                 

PERFORMANCE TESTS (continued)                                                                                    

                                                                                                                 

                                                                             The performance test subgroup       
                                                                             storage_reclamation_implicit        
                                                                             (SR_IM*) may crash every time.      
                                                                             Skip the rest of the subgroup, or   
                                                                             be prepared for multiple system     
                                                                             reloads.                            

                                                                                                                 

8. Warning message in log preceded by  The compiler may have performed an    Examine the code generated for the  
string "@@@" stating that a test       unexpected optimization, or there     test problem to see whether the     
problem not anticipated to be NULL     may have been contention when         compiler has recognized the test    
was executed in zero (or near zero)    running ZG_INIT (or initialization    problem as being invariant with     
time.  These cases are also reported   code in ZG_INLNE) resulting in too    respect to the timing loop.  A      
by SSA.                                large a value for LOOP_TIME.          compiler with interprocedural       
                                                                             optimization may have optimized     
                                                                             this test problem out of the        
                                                                             timing loop and also moved parts    
                                                                             of other test problems.  ALL        
                                                                             measurements made with this system  
                                                                             are   suspect   until   this        
                                                                             condition  is resolved.   Suggest   
                                                                             user  add  a call on an assembler   
                                                                             routine to ZG_GLOB3.PROC_SPOIL.     

                                                                                                                 

9. Warning message in log preceded by  ZG_INIT may have been run with        Rerun ZG_INIT without contention    
string "@@@" states that a test        contention, resulting in too large a  to see if the value of LOOP_TIME    
problem with a code expansion size     value for the loop overhead time,     becomes smaller.                    
greater than zero executed in zero     making all measurements that were                                         
(or near zero) time.  These cases are  made using this value of LOOP_TIME                                        
also reported by SSA.                  too small.                                                                

                                                                                                                 

                                                                                                                 

               SYMPTOM                                PROBLEM                             SOLUTION               
                                                                                                                 

PERFORMANCE TESTS (continued)                                                                                    

                                                                                                                 

                                       The target processor may be           Examine a listing showing the code  
                                       overlapping execution of problem      generated for the test problem.     
                                       with timing loop code.                If code is executed each time       
                                                                             through the ACES inner timing       
                                                                             loop, there is not a problem.       
                                                                             Some target processors really can   
                                                                             execute some instructions in zero   
                                                                             incremental time.                   

                                                                                                                 

                                       The test problem may have been        Examine a listing showing the code  
                                       validly optimized into a NULL but     generated for the test problem      
                                       the size measurement may be invalid.  (and the loop overhead in ZG_INIT)  
                                        If the compiler generated larger     to see if this occurred.            
                                       code for the (null) loop in the                                           
                                       ZG_INIT routine than for the                                              
                                       (optimized into null) loop in the                                         
                                       test problem, all the size                                                
                                       measurements are suspect.                                                 

                                                                                                                 

ANALYSIS:CONDENSE                                                                                                

                                                                                                                 

1. CONDENSE issues a message that      Overhead measurements must be         See Section 9.3 "CONDENSE" in the   
compile and link time overhead         subtracted from compile and link      User's Guide.  Collect overhead     
measurements are not present in log    times.                                measurements and add to beginning   
file.                                                                        of log file.                        

                                                                                                                 

                                                                                                                 

                                                                                                                 

                                                                                                                 


               SYMPTOM                                PROBLEM                             SOLUTION               
                                                                                                                 

ANALYSIS:CONDENSE (continued)                                                                                    

                                                                                                                 

2. Analyze cmp_time database created   When cross-checking execution and     User may manually select the cmp    
by CONDENSE contains an                compile time data, CONDENSE found     time by uncommenting it in the      
"inconsistent" error code for a file   that the execution results for this   database.  User must decide: 1)     
or program name.                       test were error codes, or were a      Should compile times be used when   
                                       mixture of error codes and valid      the execution failed 2) which of    
                                       data, so that it was not possible to  several compile times is valid if   
                                       choose among several compile times.   some executions were successful     
                                                                             and some not.                       

                                                                                                                 

3. CONDENSE will not pick up compile   Labels on times are not compatible    Check labels output in command      
and/or link times, or lists them all   with those expected by CONDENSE, or   files.  See User's Guide Section    
as "partial."                          the labels are not in the sequence    9.2.2 and the Reader's Guide        
                                       expected.                             Section 5.2.1 "Input and Output     
                                                                             Files".                             

                                                                                                                 

CAPACITY TEST:GENERAL                                                                                            

                                                                                                                 

1. Test completes with a time limit    Test sizes are too large.             Decrease size of input minimum      
exceeded indication.                                                         and/or maximum, or increase test    
                                                                             time.                               

                                                                                                                 

CAPACITY TEST:COMPILE TIME                                                                                       

                                                                                                                 

1. Output messages cease while test    Generated program may be too large    Decrease size of test input         
is running.                            to compile or run.                    parameters.                         

                                                                                                                 

2. Erroneous results produced when     File "yc_inter.txt" is being          Run only one yc_drivr at a time.    
two versions of "yc_drivr.com" are     corrupted by the other process.                                           
invoked concurrently.                                                                                            

                                                                                                                 

               SYMPTOM                                PROBLEM                             SOLUTION               
                                                                                                                 

CAPACITY TEST:COMPILE TIME                                                                                       
(continued)                                                                                                      

                                                                                                                 

3. Message ">>>error in input test     First string of nonblank characters   Correct the test name in            
name".                                 on the corresponding test parameter   "yc_parms.dat".                     
                                       line is not in the form of CTnn                                           
                                       (where CTnn represents Compile Time                                       
                                       Tests CT01 through CT32) followed by                                      
                                       a blank.                                                                  

                                                                                                                 

4. Message ">>>error in input          Starting test number is too large,    Replace with a valid number in      
starting test number."                 negative, or is not in a valid        "yc_parms.dat".                     
                                       integer format followed by a blank.                                       

                                                                                                                 

5. Message ">>>error in input ending   Ending test number is too large,      Replace with a valid number in      
test number."                          negative, greater than starting       "yc_parms.dat".                     
                                       number, or is not in a valid integer                                      
                                       format followed by a blank.                                               

                                                                                                                 

6. Message ">>>error in input test     Time is negative, greater than 24     Replace with a valid number in      
time."                                 hours, or is not in a valid floating  "yc_parms.dat".                     
                                       point format followed by a blank.                                         

                                                                                                                 

7. Message ">>>error in input test     Test description string is missing,   Add test description string of      
description."                          or greater than 60 characters.        appropriate length to               
                                                                             "yc_parms.dat".                     

                                                                                                                 

8. Message ">>>error : missing         Not enough input parameters on test   Check to see that the test name,    
parameters in yc_parms.dat."           parameter line.                       test minimum, test maximum, test    
                                                                             time, and description string are    
                                                                             present in yc_parms for the         
                                                                             corresponding test.                 

                                                                                                                 

               SYMPTOM                                PROBLEM                             SOLUTION               
                                                                                                                 

CAPACITY TEST:COMPILE TIME                                                                                       
(continued)                                                                                                      

                                                                                                                 

9. Message ">>>error : unexpected      Unexplained error in the program      See User's Guide Section 10.1,      
exception in program yc_xxxxx" where   "yc_xxxxx.ada".                       "How to Submit a Problem Report".   
"xxxxx" is the program identifier.                                                                               

                                                                                                                 

10. Disk space keeps shrinking.        Ada library is heavily used.  A       When space becomes too small,       
                                       likely problem  if  the  system       delete library, recompile the       
                                       under test does not have a "delete    globals and other dependent         
                                       unit from library" capability.        packages, and restart the tests.    

                                                                                                                 

CAPACITY TEST:RUNTIME                                                                                            

                                                                                                                 

1. Message ">>>error in test name for  The first string of nonblank          Correct the test name or add the    
rtnn" where rtnn represents test       characters for the appropriate test   parameter line for this test in     
numbers rt01 through rt09.             does not start with rtnn.             "yc_parms.dat".                     

                                                                                                                 

2. Message ">>>error in input          Starting test number is too large,    Replace with a valid number in      
starting test number for rtnn."        negative, or is not in a valid        "yc_parms.dat".                     
                                       integer format.                                                           

                                                                                                                 

3. Message ">>>error in input ending   Ending test number is too large,      Replace with a valid number in      
test number for rtnn."                 negative, greater than the starting   "yc_parms.dat".                     
                                       test number, or is not in a valid                                         
                                       integer format.                                                           

                                                                                                                 

4. Message ">>>error in input test     Time is negative, greater than 24     Replace with a valid number in      
time for rtnn."                        hours, or is not in a valid floating  "yc_parms.dat".                     
                                       point format.                                                             

                                                                                                                 

5. Message ">>>error in input test     Input description string is missing   Add description string of proper    
description for rtnn."                 or more than 60 characters.           length to "yc_parms.dat".           

               SYMPTOM                                PROBLEM                             SOLUTION               
                                                                                                                 

CAPACITY TEST:RUNTIME (continued)                                                                                

                                                                                                                 

6. Message ">>>error : missing         Not enough input parameters on test   Check to see that the test name,    
parameters for rtnn."                  parameter line.                       test minimum, test maximum, test    
                                                                             time and description string are     
                                                                             present in yc_parms for the         
                                                                             corresponding test.                 

                                                                                                                 

7. Message ">>> error : unexpected     Unexplained error in the program      See User's Guide Section 10.1,      
exception in program yc_rtxxx."        "yc_rtxxx.ada".                       "How to Submit a Problem Report".   

                                                                                                                 

8. Message ">>> error: exception in    Possibly a storage error due to       Reducing the test maximum may       
procedure process_status."             large test size.                      eliminate this.                     

                                                                                                                 

9. yc_rt07 does not produce a results  By the nature of this test if the     The test results summary has to be  
summary.                               test fails because the system memory  created from the output test log    
                                       limit was exceeded the only output    information.                        
                                       is the log information.