Adding SPICE while preserving the CMM
Paul Rogoway Motorola Corporate Director Software Quality Standards 16 Kremenetski St. Tel Aviv 67899 ISRAEL bpr004@email.mot.com Tel: +972-3-565-8950 Fax: +972-3-562-4925
After a brief description of the background, we discuss the overall software process improvement paradigm and then propose a strategy for achieving maximum improvement at minimum cost.
Many factors have been suggested as to why the CMM approach seems to be "necessary but not sufficient". One of the primary contentions is that the CMM does not address many crucial processes or areas of activity. The SEI has recently attempted to address this problem to some extent by producing a version 2.0 [3] with expanded Key Process Areas, especially at the higher maturity levels 4 and 5. That effort has now been folded into a broader effort to address systems engineering and integrated product and process development considerations, called the CMM Integration Project, or simply CMMI.
The CMMI Project is a joint effort of the United States Department of Defense (DoD), the SEI, and industry. Unfortunately, this project has gotten off on the wrong foot because SEI's new sponsor within the D0D, the Acquisition and Technology Department (A&T), was felt by many members of the CMM user community to be too heavily focused on systems engineering and the U.S. defense community, at the expense of moving as quickly as possible toward improvements that would affect the software development organizations in commercial industry and other non-military organizations worldwide. Once made aware of these fears, the DoD responded positively by inviting commercial companies to join the Stakeholders/Reviewers Team, by committing to release a software-only CMM as the first CMMI product, and by declaring intent to work toward acceptance of CMMI as an international standard. Meanwhile, though, the CMM v2.0 release was delayed, its Advisory Board was deactivated, and the CMMI Steering Board still does not include commercial industry representation. Furthermore, the CMMI task has turned out to be very substantially larger than originally anticipated, so the schedules are fluid at this time.
Because of the above, it behooves organizations who have invested substantially in the (software) CMM as the basis for their SPI programs to explore ways to protect this investment, e.g. if the CMMI Project turns out not to provide the complete answer to these organizations' needs.
SPICE represents a strong possibility to assist in meeting the needs of the CMM community with minimum waste of previous investment, as discussed below.
The Software Process Improvement paradigm is described in the following SEI "IDEAL" model:
[Figure 1]
In SPICE terms, the diagram looks as follows:
[Figure 2]
As can be seen from both diagrams, the appraisal or assessment is just one of many links in the chain. This suggests that an organization with a well understood and mature improvement approach currently using the CMM for assessment is likely to want to retain as much as possible from its current improvement approach while aiming to incorporate as much as possible from SPICE in an evolutionary way. It is true, however, that the process framework underpins the entire improvement cycle, and must therefore be given very strong consideration.
As it turns out, both the CMM and SPICE do not provide prescriptive models, so one can indeed entertain the thought of transitioning in a relatively smooth manner. While the CMM does include key practices for every Key Process Area (KPA), the criterion for determining an organization's capability maturity with respect to a given KPA is whether the organization has satisfied or achieved each of that KPA's goals. If this occurs by virtue of having executed each specific key practice, fine; but equally valid is the execution of alternative practices shown to achieve the given goal(s). IN SPICE, the situation is even more flexible. Part 2, which is normative (required), defines a reference model with a process dimension (a set of processes) that specifies only the purpose and expected outcomes of each process. Part 5, which is informative (guidance) only, is an example of an assessment model which includes an example set of indicators of process performance and capability.
In both the CMM environment and in SPICE, there is also flexibility with respect to the assessment method employed. The SEI provides a CMM-based Appraisal Framework (CAF) which defines the rules for an acceptable method, while SPICE defines the conformance criteria for both the assessment method and the assessment model; So long as a given model and method conform, the assessment is deemed to be SPICE conformant. SPICE trials using a variety of methods have produced promising results with respect to consistency of findings, positive evaluation by assessors, and applicability across a wide variation in organization size and type.
Due to limited time and space, we will not dwell on the issue of transitioning a given assessment method. Suffice it to say that there is enough encouraging information to indicate that the transition for an organization using the CMM today to a SPICE conformant assessment method is likely to be relatively easy. In the rest of this article, we will focus mostly on transitioning the processes i.e. the assessment model.
Given that we do have some concerns about the current status of the CMM world, why should SPICE be of interest? The answer is that SPICE and ISO/IEC 12207 [4], upon which its Process Dimension is based, provide a great deal of additional functionality. Briefly, a summary of a comparative evaluation is as follows.
Since 15504 is a proper superset of 12207 and since a revision to 12207 is now planned, which will include the 15504 additions, we can focus on 15504 for our purposes.
So, just how do we go about incorporating these additions into our organization without throwing out the enormous investment that we have made in the CMM as the basis for our entire SPI program? The answer is, we need to move in an evolutionary way, from point A to point B, as described below.
Produce
a 15504-Conformant CMM
Let's stick with the CMM, well accepted and well understood in our organization, and add to it the 15504 enhanced features. We probably will want to start with the CMM 2.0 because it does include many of the additions in the spirit of the current CMM, and a few name changes in the right direction. Because we are talking about quite a few additions, it will be up to each organization to decide whether to add these all in one step or in two or more steps. When we are through, we must have a model which is conformant with, i.e. mappable to, the 15504 Process Dimension.
As a corollary to this part of the plan, we also ought to try to work toward as close compatibility as possible between the CMMI Project's software-only CMM and our 15504-conformant model. The way to do that is to try to influence the CMMI Project as much as possible in that direction while bending as much as possible to adopt whatever they produce. It is encouraging to note that one of the expressed goals of the CMMI Project is to attain international standardization status for its products.
Update the CAF / CBA-IPI accordingly to be SPICE-Conformant
We won't belabor this point, but we do need to ensure that we also define and use a SPICE-Conformant assessment method. Fortunately, as mentioned earlier, the CBA-IPI is already quite close to conformant, and it should be a relatively straightforward task to make any necessary changes and demonstrate conformance. Whether to adjust the CAF as well will depend at least partly on what the CMMI Project does in this direction and on the extent to which alternative methods turn out to be in use in the CMM user community.
Migrate to 12207/15504 terminology and "fragrance"
Once our model is in widespread use and the 12207/15504 situation has stabilized, we probably will want to move closer toward that ISO "flavor" or "fragrance". This decision will not be taken lightly and must be made taking into account many factors including the extent of usage of the CMMI products and 12207/15504 based products, availability of tool support, expected return on investment, etc.
An exciting benefit of taking this step is that the basic assessment model can serve as the basis for the organization's actual processes. The CMM KPAs, as is well known, are not intended and not really appropriate for use as processes by an organization for development of software. Usually IEEE 1074 or ISO 12207 r some other model is used. But since 15504 is itself based on 12207, the assessment model may indeed be appropriate to serve both needs.
Ideally, this change would be institutionalized at an optimum time for the organization, e.g. when things are particularly quiet, or when major change is occurring which could readily include the migration. Training and related activities must be planned and implemented thoroughly.
Migrate to a continuous version of the model, if suitable
Depending again on how the world moves, as well as on the organization's culture, migration to a continuous version of the model should be considered. The time to do this might be as part of the migration toward 12207/15504 terminology, or separately. It should be noted that this issue is a major one, and the CMMI Project intends to produce staged models while also providing continuous models along with guidance on how to use them.
Continue improvement, e.g. by incorporation of CMMI products
As organizations improve, they will naturally want to increase the scope of use of capability models throughout the organization. Assuming there is not an unconquerable barrier between 12207/15504 and CMMI, it would seem most appropriate to seek extension of the process improvement banner across the organization by adopting (and possibly adapting) the CMMI approach and models to disciplines other than software.
[2] ISO/IEC TR 15504-1:1998 Information technology - Software process assessment - Part 1 : Concepts and introductory guide (Also, Parts 2-9)
[3] Software Capability Maturity Model, Version 2.0, Draft C (not released), Pittsburgh, SEI, November, 1997.
[4] ISO/IEC 12207: 1995 Software Life Cycle Processes
[5] Rogoway, Paul: SPICE - Proposed International Standard on Software Process Improvement Determination, International Conference on Quality, Jerusalem, ISRAEL, November, 1996.
[6] Rogoway, Paul: Transitioning to SPICE, SPICE96, Brighton, England December, 1996.
MOTOROLA EXPERIENCE (16 Years)
Serves on Corporate Software Engineering Technology Steering Committee and on various international committees and working groups in areas such as software life cycle models, capability assessment, technology and tools, and process improvement. Israel's delegate to several international software standards organizations, including those responsible for SPICE and ISO 9000-3. Chairperson of the Israel Software Process Improvement Network (I-SPIN).
Member of Motorola's Science Advisory Board Associates (SABA). Winner of Motorola Outstanding Impact Award for contribution to ISO 9000-3 revision project. Authorized Lead Assessor in the SEI Appraiser Program.
Previously established and managed two software development groups, headed a Motorola Senior Executive Program team which produced a methodology for software engineering technology planning, founded and managed "4S" to help Motorola and non-Motorola organizations accelerate their improvement in software development capability, served as process improvement coach/consultant to several Motorola software development organizations and to leading non-Motorola software development organizations in Israel, and was responsible for the first phase of the CASE* project to define an Integrated Project Support Environment for Motorola worldwide,
More than 40 papers, articles, and lectures on various software engineering topics
Motorola is a multinational company whose products include semiconductors, cellular systems, paging systems, two-way radio communications, modems and integrated management systems, automotive electronics and software, government and space systems, and multimedia
Within Motorola, products and systems are becoming software intensive at a dramatic rate. Every operating unit is experiencing a growing dependence on software development - internal and external. Engineering assets are rapidly becoming software intensive.
Software Process Improvement, including Self-Assessments, has been in use in Motorola since 1989; there have been over 200 assessments, and there are over 200 trained assessors. Self-Assessment methods used include
SPA, CBA-IPI, CMM based self-evaluations (SW-CMM, SE-CMM, P-CMM),and, as of 1997, Motorola's proprietary CMM-based CAF compliant self-assessment which is part of the Quality Systems Review (QSR).
Assessors are volunteers from across business units.
Motorola's historical Involvement with SEI began with very early use of self assessments in 1989. The following year SEI trained Motorola assessment trainers who have since then been training other Motorola assessors. Motorola persons have held membership on the CMM-Based Assessment and Software CMM advisory boards, and a Motorola until recently was the chair of the Software CMM advisory board. Motorola have also been active participants in the People CMM development and in piloting of models and methods, including the SW-CMM (V1.1, V2.0), P-CMM, SE-CMM, IPD-CMM
SPA, CBA-IPI. We were also represented as a Member of Board of Visitors (1996).
Some of the important milestones are as follows:
1989 First SEI self-assessment
1990 Selected SW-MM as model of choice
1990 Embraced Level 3 goal for Motorola
1991 "Software Solution" initiative
1993 First Level 5 organization (now have several Level 5 organizations)
1995 >75% SW product population at L3
1997 Set Level 5 goal for company
Since 1992, Motorola has achieved the following: More than 75% of Product Development is at Software Maturity Level 3 and above, there has been a
7X software quality improvement rate in the last two years (now at ~5.7 sigma quality), there has been greater than 3X productivity improvement, and a 3X software cycle time improvement. This record clearly places Motorola at the forefront of the international Software Process Improvement community.