Section 6 Choosing Software Development Products This is the second SASEA subprocess. It recognizes that a wise language choice cannot be made without consideration for tool sets available for development in that language. This section discusses the selection of a tool set in detail. Cost Considerations Once the language choice is completed, the next consideration usually must be cost. Budget constraints will certainly determine the candidate products to be considered. However, the purchase of software tools does not usually cover all of the cost items that need to be considered. Table 4 provides a cost worksheet for listing most of the possible cost items associated with purchasing development products. A blank row has been provided for adding other possible costs. The cost items listed in Table 4 cover the various types of costs that may be associated with the choice of particular development products. Each column represents a tool set to be considered. If one product will satisfy all of the needs for development tools, including design tool, metrics tool, editor, compiler, etc., then it should be considered in one column. On the other hand, if it takes two or more products to cover all of the required tools, then the costs of a complementary set of products should be combined in one column. This makes for a clear, direct comparison between the costs in the various columns, and it will help in making the final choice of product(s). Table 4: SASEA Cost Worksheet Candidate Product Name Cost Factor             Purchase of product             Maintenance on product             Training             Installation             Additional hardware needed             Additional software needed             Additional people needed                           Total Cost of Product             The number of columns in Table 4 in no way represents the number of candidates that should be considered. This table should be expanded to include as many products as an organization wants to consider. The cost factors in Table 4 are defined in Appendix E, along with a discussion of why they are important to the cost considerations. The costs listed in a particular column in Table 4 should be the actual cost to the organization to properly prepare it for using the product(s) listed in that column. In other words, if a tool is already owned by the organization, then its purchase price should not be included. However, additional operational costs, such as licensing more users, may need to be considered. If the hardware required to run the software tools already exists in-house, that cost is not included, but the cost of any hardware that must be purchased to use the software development tool(s) is included. If additional software personnel will be hired for the project regardless of the tools used, that expense is not included. However, if extra people are required specifically to make it possible to use the product(s) in a particular column, then that expense is listed in that column. Filling in the costs in Table 4 may be a time-consuming task. It will require sufficient investigation of each tool to reasonably estimate the cost involved for a particular cost item. Where a cost is in doubt, begin by assuming the lowest possible cost so any error will result in a tool continuing to be a candidate rather than it being eliminated prematurely. After the product selection process described below, the costs will be reconsidered in the light of increased knowledge about the real costs required to use each product. At that time, errors of underestimating expenses will come to light, and products too costly can still be eliminated from consideration. The total costs (in the last row) resulting from completely filling out each item of Table 4 will provide a preliminary picture of the relative total cost to the organization for using each set of tools represented in each column of the table. This permits the decision maker to pare down the list of candidate tools for continued consideration. The next step will require considerably more effort in evaluating each product that is still a candidate, so it is desirable for this list to be as small as possible. However, all products that are viable candidates should still be included, regardless of number. It is important to recognize that the costs considered in this preliminary cost analysis are only those costs directly related to the software development products to be used by the organization. The actual development process will include many more costly items, such as the cost of personnel assigned to software development over the life of the project. To put cost in proper perspective, over the lifetime of a software system the maintenance of the software will usually cost 2 to 4 times the total cost (software, hardware, people, etc.) of developing the system in the first place [Schach 93] [Sommerville 89]. This is why language characteristics, such as maintainability, are so important. Any factor that can support the reduction of the cost of maintenance, and/or the extension of the lifetime of the system, will contribute to a very significant cost/benefit improvement for the system, far in excess of the cost of setting up the software development tools. Choosing the Tool Set The SASEA decision process is not complete until a product or set of language products is chosen to use for the software system development. Table 5 lays out the SASEA subprocess for selecting language products. Each row represents a product characteristic important in many such decisions. Note that cost value is a characteristic, allowing cost to be a factor in product selection (a high cost value rating meaning low cost and a low cost value rating meaning high cost). Other characteristics are organized as performance, tool support, or usability characteristics. The shaded boxes are only intended to show the organization, while the boxes in the rows for particular characteristics are intended to be filled in where applicable. The Weight column is for weighting the importance of these characteristics to the organization or project. As in Table 2, weights should range from 0 to 10, where 0 means no significant importance, and 10 means critical importance. The remaining columns are for the candidate products from Table 4 that have survived the scrutiny of cost constraints. The same product or set of products used in a column in Table 4 should be placed in one column in Table 5. Add as many columns to Table 5 as required to accommodate all of the candidate tool sets. The product characteristics in Table 5 are defined in Appendix F, along with an explanation of how each characteristic affects the software development process. Once weights have been assigned to each of the product characteristics in Table 5, the next step is the most difficult and time-consuming of all. For each non-zero weighted characteristic, a rating must be determined for each column of candidate products. This rating should be in the range of 0 to 10 (as in Table 2), where 0 means the characteristic is not supported at all by any of the products in the set represented by that column, and 10 means outstanding support for that characteristic by the represented tool set. The amount of time, effort, and cost spent in the product evaluations for Table 5 should be representative of the expected overall cost (from Table 4) of the tool investment. It makes no sense to spend a staff-year on a purchase of around $10,000 that will only be used for one project. On the other hand, evaluations of sets of tools that may cost more than $100,000, and be used on multiple projects involving hundreds of staff-years and millions of dollars, are well worth the expenditure of significant resources. Table 5: Ratings of Language Product Characteristics Candidate Product Name   Product Characteristic W e i g h t           Cost value             Performance             Compilation speed             Execution speed             Ability to handle large programs             Tool Support             Requirements specification/ Risk analysis             Design             Code generation             Documentation             Interfacing with a database             Testing             Static analysis             Dynamic analysis             Traceability analysis             Configuration management             Interfacing with other languages             Usability             Ease of set-up             Ease of use                           Overall Product Rating             It may be possible to obtain evaluation information on some tools and tool sets from outside sources. This is very desirable, but some cautions apply. To be valid, evaluation information must be recent, and it must apply to the same configuration of tools and the same use of the tools as for those represented in Table 5. One must also be wary of the sales brochures which usually exaggerate tool capabilities and often use incorrect terminology. More objective data is required. Once all ratings are filled in for the non-zero-weighted characteristics in Table 5, the overall product ratings (last row) can be calculated using the Rating Formula, as explained for Table 2. Again, round the overall result for a product set, Ri, to one digit. As with Table 2, the rounding step is important to avoid being misled by false precision. Also as with Table 2, ratings determined by using different characteristics and/or different weights should not be directly compared. The normalized overall product ratings will be in the range of 1 to 10, and the best products will be those with ratings nearest to 10. The range of actual ratings will depend to a large extent on the rating patterns of the raters. If no product set receives an overall rating that would be considered acceptable (for example, if 6 is considered acceptable, but all products are rated less than 6), then it may be necessary to reexamine the choice of language. However, it is advisable to first reexamine the biases and rating patterns of the raters, and also to reexamine the breadth of products being considered. The SASEA decision process is complete when the overall product ratings (last row of Table 5) are used to select the tool set to be used for software development. This could be a very simple decision, if one product rates significantly higher than all others. However, it may not be that easy. If multiple product sets have similar ratings, a sensitivity analysis can be performed to see if varying the assigned weights in the table makes a significant difference in the overall ratings. Since the weights are only approximations at best, vary them somewhat and see how this affects the final product ratings. If one product set is not clearly superior, cost may be reconsidered at this point. With the increased knowledge of the products gained by evaluating them for Table 5, the costs for the products still being considered should be updated in Table 4 (removing the columns for products no longer under consideration will reduce confusion). It is now up to the decision maker to determine which tool set is the most cost-effective for meeting the organization’s software development requirements. It should be noted that Tables 2 through 5 support a decision to be made for a specific project of interest, but the process also supports the long-term cost-effectiveness of the decision for supporting the organization’s business plan.

Sections

1

2

3

4

5

6

7

Appendices

A	B	C	D	E	F	G	H	J
K	L	M	N	P	Q	R	S