Ada Used for
Distributed Process Control
at Weirton Steel
Hot Mill Automation
Systems Process Engineer
Senior Analyst, M.I.S.
Background and Process Overview
- The competitive tolerances that the control system should maintain for key product parameters (strip width, strip thickness, strip shape/crown, finishing temperature, coiling temperature, and surface quality.)
- Functional requirements for mill control and quality reporting.
- A required functional hardware hierarchy and distributed architecture that stressed hardware redundancy and flexibility as well as software unit functional independence and reliability.
- The requirement that supplied software be operating system revision independent, so that Weirton would not end up frozen with outdated and unsupportable hardware and system software.
- Minimum levels of spare capacities to allow for anticipated system enhancements.
- A general philosophy on operator interaction to the system.
- Required diagnostic packages and software alarming.
- Scope of software supply including documentation and source code supply requirements.
- A general description of the participation of Weirton's engineers during software design, coding, and testing at the vendor's factory.
- Minimum requirements for system functionality, reliability, and performance testing at the vendor's factory.
- A description of the statistical analysis to be used in evaluating product quality performance.
A distributed architecture was requested only after many hours of deliberation, since the standard for the steel industry up until that time was a centralized system, with all functions performing by the real-time high level language (product tracking, mill setup modeling, operator interfaces, reporting, diagnostics, etc.) resident in one large computer, usually a DEC VAX. Although a distributed system had been untried in the steel industry, the following advantages led to the decision to proceed in that direction:
- Distributed systems facilitate hardware sparing.
- Weirton's requirement for phased project implementation would be simplified and have some flexibility from the system's perspective.
- System expansion provisions become more simplified and flexible.
- Hardware redundancy and reliability requirements could be met more economically.
The specification also stated that application software should be written in either Fortran 77 or Pascal using structured programming techniques. Reasons for this request included:
- The vast majority of Weirton's process control and real-time quality reporting projects up until that time had been written in these two common languages.
- Weirton management felt that Weirton's internal process control support staff would be most effective maintaining and upgrading a system written in industry standard higher level languages.
- Weirton management felt that system support would be more easily available for a system written in widely accepted languages.
Since the Hot Mill was operated in a totally manual method before this project, Weirton also requested the electrical supplier to supply manual operational capabilities for each functional area that were to be used to keep production flowing during problems with any of the control computers. From Weirton's perspective at the time, manual operation looked to be a viable option. Looking back, this was a major design flaw since, after the first month of operation, it became apparent this production unit would never run manually again. The complexity that was added to the system to allow for manual operation turned out to be a large waste of engineering, hardware, and software development resources.
Since the electrical and automation project was to be supplied as one complete package (electric motors through supervisory computers), potential vendors were limited to the world's major electrical suppliers. Bids were taken from European and American suppliers.
The overall system specification allowed for flexibility on the part of the supplier, which meant changing or relaxing some detailed requirements to facilitate competitive bidding. This is how the Fortran/Pascal specification was relaxed to include Ada as a choice. Weirton was against the use of Ada since we had no prior experience with Ada and the Hot Mill project was critical to the corporation's economic survival. General Electric was in the process of supplying a distributed hot mill control system for an overseas concern using Ada that was very similar to what the Weirton specification called for. After a long period of discussions, the specification was changed to permit Ada. During these discussions, General Electric made the following arguments favoring the use of Ada:
- Ada is self documenting.
- Software written in Ada will run if it compiles.
- Ada inherently gives program modularity.
- The ELN operating system General Electric planned to use within their distributed system limited languages to Ada, Pascal, and C.
- Experienced General Electric software engineers extolled the virtues of Ada.
- General Electric enumerated the economic advantages of using their "off the shelf"' Ada software as a starting point for our project.
As it turned out, the programming staff at General Electric proved to be a great help in system maintenance because Weirton's software is not a totally one-of-a-kind system.
After the selection of General Electric as the supplier and the project got under way, Weirton sent a 10-person team to work at the General Electric Drive Systems Division. This team eventually spent 20 consecutive months at the General Electric factory. Their first assignment was a three-week course in Ada taught by Digital Equipment Corporation. It was then the responsibility of the Weirton team to ensure the system was developed and tested in the manner dictated by our design specification. Most of Weirton's key software engineers had both real-time Fortran and Pascal experience, and the transition to Ada was made smoothly.
The use of Weirton personnel from the beginning of the software lifecycle ensured continuity from the design and analysis through system maintenance. Weirton personnel were assimilated into the General Electric software development team and took direction from General Electric supervision. Weirton personnel were also assigned to develop software modules (packages and tasks) that would be used in the system. The Weirton system and software development team had a positive influence in the automation project for several reasons:
- Team members were able to make all parties involved (management and engineering personnel from both Weirton and General Electric) better understand the nature of the problems which made the design phase go much smoother.
- During the design phase Weirton personnel learned how General Electric approached system development, from coding through system testing.
- Communication between the team and Weirton's existing Hot Mill personnel has afforded our operations and maintenance personnel a more complete understanding of the system.
- The Weirton personnel played an essential part in the testing of the system. It should be noted that several other hot strip mill projects were shipped before Weirton's, even though they were started after we started, because our system was undergoing such extensive testing.
- points to the originating file for the source code;
- keeps a copy of each unit's object code;
- keeps a copy of the source code;
- provides hierarchical library structure;
- keeps track of obsolete units;
- submits units for linking.
Packages to be used system-wide were developed early in the project and reside in the Ada library SYSSADALIB (for example, the package defining the data types uses for physical units measurement). Packages to be used by many applications that are dependent on system-wide packages reside in the Ada Library MAINT$ADALIB, which is a sublibrary of SYS$ADALIB. Each major application has its own Ada library, each of which is a sublibrary MAINT$ADALIB.
The total development environment also relies heavily on the VMS directory structure. Each major software function has its own
directory structure, with the major functions broken into modules which also have their own directory structure. Each directory structure consists of areas for source code, documentation, executable code, and development. The software for the Hot Mill system is, by definition, an embedded control system. The software employs parallel processing, real time control, unique I/O control and handling, exception handling and fault tolerance. The nucleus of Weirton's system is the VMS/ELN operating system, which is surrounded by General Electric system tools. These tools are standard for most General Electric systems and perform functions such as drive systems control, input and output collection and distribution, inter-task communications, finite state machines, pre-processors, and tune-up and debugging aids.
An outer shell of application influenced or application specific tasks and procedures perform functions such as communications to the business system, data logging, reporting, and maintenance and diagnostic tools. Note that the lines between these functional layers is not always obvious.
As mentioned previously, all of the Level II control system application software and most of the system tools are written in Ada. Some of the standard General Electric tools provided that were carried over from previous General Electric supplied systems were provided to Weirton in Fortran. In addition, the model that provides setup and control of the finishing mill is written in Fortran. Early in the development of our system, it was decided that there would not be sufficient time within Weirton's development and installation schedule to translate the existing finishing mill setup models into Ada. Since the finishing mill model resides in the finishing ELN node, an Ada shell was developed provide an Ada interface between other software and the model. This Ada shells links the models as foreign procedures within the Ada program. The Ada shell provides an outer level exception handler to the Fortran model. Note that the roughing mill models installed in Phase II are object-oriented and are written in Ada. Within Weirton's control system, there approximately 40 megabytes of Ada application code, including comments.
Ada in System Design and Development
- A package of routines used by applications to access the database describing slabs scheduled to enter the Hot Mill process.
- A package of routines providing product tracking applications the functions of adding, deleting, or moving in-process product.
- A package of routines that provide access to the extensive memory resident data records for each in-progress slab/strip.
- A package providing applications the necessary routines for reading and writing in communication with Level I applications via the CSF high-speed data highway.
- A package providing applications the necessary routines to generate alarms for mill operators and/or engineering use.
- A package of routines used by control software residing on the different physical computers to communicate real-time.
- A package of routines used by the real-time tracking routines to define and do appropriate processing for changes of state to specific data defined on the CSF high-speed data highway.
- Packages used within the Ada models that define physical mill components as data objects, providing flexibility for a necessary changes to the physical mill.
The common code and standard packages upon which Weirton's system is built form a baseline that, if grasped by the program developer or maintenance engineer, provide an understanding of the building blocks for almost all of the applications within the control system.
Two types of Ada tasking are used in the control system. The five separate and independent product tracking systems are started once and must remain running under all circumstances. Each communicates with the others via inter-tracking communications. Other tasks are spawned to do a particular job, such as logging or data transfer, and then terminate.
Generic units are used extensively to carry out database, input/output reading and writing, along with other services such as generic train services.
The error building features of Ada have proven extremely valuable in Weirton's control environment. Due to the control aspects of the system, most of the tasks must remain running under any and all conditions. Therefore, error handing must be comprehensive, predictable, and allow for the reliable recovery of the routine experiencing the error.
Process problems are handled with a "raise" within the Ada code that transfers execution error handling as any control system error would. For example, a slab that is too wide to be processed raises an exception just as a zero divide error does. Ada allows the Weirton system to supply predictable outcomes to extraordinary conditions that result in control system errors The "exception" area of the code is used to generate engineering and/or operator alarms, supply default data values when necessary, and to continue processing.
The strong data typing features of Ada have been utilized within the system to reliably constrain input and process variables. Each variable of the primary data input record for a slab scheduled to be rolled is "typed" against critical limits for that particular variable. For example, slab_ width_ type denotes a range from 24 to 50 inches [61 cm and 1.27 m]. If the input value causes an exception to be raised, a default value can be used. In cases where a default is not permitted, the actual input value is used and appropriate operator alarms are generated.
Ada was beneficial during the testing of the control system. Simulations were done to test a variety of operating conditions. Interactive software was written which simulated system process inputs. Each simulated slab is represented by a separate task. Multiple slabs being processed concurrently are simulated by starting multiple tasks.
Development of the system was noticeably more programmer independent than utter control projects the Weirton team has been involved with. During software development for a particular function, once the interface design work is complete, work on "separates" that perform more specific functions can be carried out by several different programmers.
Although system-wide standards were issued for documentation, including design documents and program abstracts, Weirton engineers have found the Ada code to be somewhat self-documenting.
Some of the drawbacks of Ada encountered by the Weirton team include:
- Changes in one of the baseline service packages forces a complete system rebuild. During development, system rebuilds were done almost daily, with compiling and task building done during nightly rebuilds and complete before the development team arrived for work. Any failures in the rebuild were reviewed first thing in the morning, with the complete system usually put back together by mid-morning.
- Strong data typing can make already complicated calculations awkward and hard to understand.
- The overuse of "uses" within applications can cause confusions within applications code. On the other hand, extensive use of the dot notation can cause an abnormal amount of typing for the software engineer.
Ada in System Maintenance
- All applications contain the same building blocks (service packages).
- Packaged services are evoked and instantiated similarly in all software modules.
- The use of consistent notation with no unnecessary differences in the modules Familiarity with the structure of one module ensures familiarity with other modules (product tracking and action routines, data collection, log generation, communications, etc.).
- Naming conventions for files, units, and variables were part of the baseline system design and are supported by the language.
- The Weirton system has extensive on-line software diagnostics. The diagnostics includes a General Electric product for on-line tuning of control variables on remote nodes, an alarm logging system together with the ability to start and stop groups of alarms within specific units of code during mill operation, and maintenance of on-line logs of text lines on the Supervisory computer that are used extensively during introduction of new functionality to the system.
- The Weirton system includes a number of tools that allow Maintenance Engineers to look closely at the process. These tools include the timely collection and storage of key process parameters on the Supervisory computer, graphics tools that allow plotting and statistical analysis of this historical data, automatic uploading of key process parameter data from the Level I controllers to an analysis package in the event of a mill wreck or "cobble," and a tool for Level I maintenance personnel to look directly into the running Level I controllers and make on-line changes if necessary.
- New applications and functionality can be added using the same service packages developed early in the development process.
- Ada in conjunction with the ELN operating system allows processes to be moved from target node to target node. This facilitates the balancing of computing power to the requirements of the software, with the addition of computing power as needed.
Since the 1992 Phase II startup, the Weirton maintenance team has also implemented a large number of major system enhancements and new control system applications, written in Ada and built upon the baseline software described above. These new functions include:
- An expanded log and data storage system in which all relevant process, system, and product data is off-line loaded to CD, and then available for analysis. During mill operations approximately 8 megabytes per hour of process and summary data are written to CD.
- Upgrades and replacement of many standard operator screens to improve user friendliness.
- Upgrades in real-time software functionality to take advantage of unique operating conditions, allowing increase mill production.
Effects of Ada on System Reliability
Possible Effects of Ada 95
[Reprinted from the Proceedings of the 13th Annual National Conference on Ada Technology with the kind permission of the authors. "Distributed Process Control Using Ada" won the conference's "Best Paper" Award in 1996.]
Conclusions and Lessons Learned