EuroSPI 2000 
Practical and innovation based software process improvement to prepare for the new millenium.
European Software Process Improvement
SPI and Systems
Category Index
Rated Newspaper Supported by EU Project 

 
 
 
 
 

Role and Impact of Feedback and System Dynamics in Software Evolution Processes and their Improvement
 

Meir M. Lehman
Goel Kahen
Juan F. Ramil
Imperial College, London, UK

Abstract

The FEAST/1 and FEAST/2 projects followed formulation of a hypothesis that apart, perhaps, from primitive processes, software processes are complex multi-loop, multi-agent, multi-level feedback systems. In order to achieve sustained process improvement processes must, therefore, be treated as feedback systems. The results of the FEAST studies (see http://www-dse.doc.ic.ac.uk/~mml/feast) include a set of models that reflect the behaviour of evolutionary attributes of real world industrial processes and their products. They also include further support for the laws of software evolution and generalisation of an approach to the development of quantitative black-box and white-box (system dynamic) models of evolution processes. Taken together they reinforce the conclusions of studies reached during the 70s and 80s. This paper summarises some of the concepts developed and lessons learnt to date, emphasising what is relevant in the context of process improvement.
 

Introduction

Software evolution includes the fixing, enhancement, adaptation and extension of a software system; that is, it encompasses all aspects of software maintenance, as generally understood. Interest in this phenomenon is now widespread. Most of the work addressing the area concentrates, however, on the means whereby the ease and reliability of evolution is increased and its cost reduced. This focus on means is generally undertaken without sound understanding of why evolution occurs, why it is inevitable, what are its key attributes and how these could be controlled and exploited to achieve a process that meets the challenges of evolution. Answers to some of the above questions emerged in a series of metrics-based studies of software systems conducted during the late 60s, and the 70s, [leh69,85]. These led, inter alia, to the laws of software evolution [leh74,85,fea00b], a concept of anti-regressive work [leh74], the SPE program classification [leh85] and a Principle of Software Uncertainty [leh89,90].
In 1993, while considering why it is generally so difficult to achieve sustained software process improvement, one of the authors (mml) hypothesised that this may be related to the feedback  nature of such processes. In so doing, he recalled a study and observation made nearly thirty years ago when, discussing the ripple observed in a plot over release sequence numbers (RSN) of the OS/360-370 growth trend [bel72], as in figure MML.0.


Fig. MML.0: IBM OS/360-370 Growth Trend

The ripple superimposed on an initial underlying linear  growth trend was then interpreted as being the result of a self-stabilising process, the consequence of counter-acting positive and negative feedback forces. On the one hand, factors such as the need to overcome limitations encountered when running the system, ambition to achieve greater market share and so on, lead to pressures to increase the functional power of the system and, hence, to system growth. On the other hand, constraints emerge that tend to limit the achievement of further growth. These include, for example, those imposed by increasing system complexity, neglect of complexity control (i.e., anti regressive [leh74] work) and resource availability. The former exemplify positive loops that reinforce a particular behaviour and so lead, in general, to growth or amplification. The latter exemplify negative feedback loops that contribute towards behavioural balance or stabilisation. This analysis led directly to the feedback interpretation referred to above. It was supported by the realisation that the instability observed after RSN 20 was related to excessive positive feedback as suggested by the well above average incremental growth between releases 19 and 20.
Recalling this early observation, he now (1993) proposed the FEAST (Feedback, Evolution and Software Technology) hypothesis [leh94]. In one of its formulations, this states that, "… with the possible exception of the less mature, software processes are multi-level, multi-loop, multi-agent feedback systems and must be treated as such to achieve sustained process improvement". The theme was discussed in three international workshops at Imperial College, London [fea94,95]. Subsequently, the UK EPSRC funded FEAST/1 (1996-1998) and FEAST/2 (1999-2001) projects have been investigating the hypothesis and its implications. More recently, a fourth international workshop, FEAST 2000, discussed related issues [fea00b].
The projects were set up to achieve a better understanding of the what and the why of evolution and to determine whether the results obtained during the 70s and 80s were applicable to the processes and technologies of the 90s. The questions being addressed included the following: How may one identify positive and negative feedback loops in software evolution processes and determine the influence of resulting system dynamics? What is the impact of feedback on such processes and on their products? How may feedback be controlled and exploited to achieve the continuing improvement of such processes and increased likelihood of meeting the challenges of evolution? In searching for plausible answers the FEAST projects have developed a set of models of industrial software processes. This work has been made possible by the provision of empirical data by industrial collaborators and their support in its interpretations. The models represent a means whereby sources of process system dynamics and their strength may be investigated. Inter alia, they can also be used to help identify potential process improvements.
This summarises some of the findings and conclusions reached to date with the emphasis on quantitative modelling in the context of process improvement. A list of references covering these and other aspects of the investigation in more details and access to them is provided via links at http://www-dse.doc.ic.ac.uk/~mml/feast.
In connection with what follows it should be noted that the scientific paradigm, as exemplified in figure MML.1, has been followed in FEAST and earlier [leh85] investigations of software evolution. A phenomenon is recognised. Metric data to describe it are obtained. Patterns are identified. Models of that data are built, analysed and validated. The observed phenomenology is interpreted in terms of the models and vice versa and expressed in a formal statement or descriptive text. Understanding of the observed behaviour, the models, and the formal statement or descriptive text are improved in an iterative process of successive refinement.


Fig. MML.1: The Investigative Paradigm


Setting the Scene

E-type software is defined as software actively used to solve a problem in a real-world domain [leh85]. The ultimate criterion of satisfaction with an E-type system is its acceptability to stakeholders. Whether it is acceptable or not will depend on their needs and desires in relation to the state of the operational domain as the system operates. But E-type operational domains are unbounded and dynamic, that is, continually changing. E-type specifications are, therefore, necessarily incomplete and will tend to become invalid. Mathematical correctness of an E-type system with respect to its specification is insufficient to guarantee stakeholder satisfaction.
It follows that continuing evolution is intrinsic to E-type software. Most of the software used in business and other organisations to support their operations is of this type. The classification is, therefore most relevant, in particular with respect to the ubiquitous need, since the beginnings of the computer era, for software maintenance. Such maintenance is, fundamentally, not due to lack of technical expertise or rigour in the development of software. It is a fact of life. Unlike the maintenance of physical equipment which is mainly targeted at restoring its original condition, the purpose of software maintenance is to maintain stakeholder satisfaction, that is to change the system to adapt it to changing circumstances, needs and desires. This difference supports the view, long expressed by one of the present authors (mml), that the term evolution is more appropriate to describe work undertaken after the first operational release of a software system than is maintenance. System evolution is inevitable and must be considered when seeking improvement of the means whereby it is achieved.
How, in practice, is system evolution achieved? As illustrated in figure MML.2, development involves, explicitly or implicitly, bounding of the application domain and definition of an application concept. Sooner or later, stakeholders converge on a common a view of the desired state of the application domain after software deployment, normally reflected in requirement and specification documentation. The latter is a de facto abstraction of the desired software in its application domain.
The precise nature or the number of individual process steps that follow is not relevant to the present discussion. The steps shown in MML.2 are one view of the activities necessary to transform an application concept into an operational program. The most important point to emerge from the figure is that, as discussed in the next paragraph, installation of the program in the operational domain, changes that domain [leh85]. Installation and operation closes a major feedback loop. Despite differences in technology, methods and process detail between the evolution processes of different organisations all share a common feature, this major feedback loop. The output from the process that is driven by the requirements and specification statements changes the application and the bounds of the domain in which it is to be pursued and which constitute the input to the process.
Once the system is deployed, lessons learned as a result of its being used, in conjunction with exogenous influences such as business, market or legislation changes, will change the application concept and/or the domain bounds. The depicted loop is a major driver of software evolution. There arises, therefore, a continuing need for software adaptation and enhancement. This is generally achieved by a sequence of software releases, versions or update as reflected in the metric and other data that record evolution of the systems [fea00a].

Fig. MML.2 : A Major Process Feedback Loop

The diagram in figure MML.2 is, however, unrealistic in several respects. Evolution processes are, in general, neither strictly sequential nor restricted to technical tasks. An improved representation is provided by MML.3. This seeks to depict the evolution process as a global multi-level, multi-loop, multi-agent process [leh94].

Fig. MML.3 : The Global Software Process

The global process illustrated is the totality, the non-linear sum, of all activities required to transform concepts, ideas, definitions, needs specifications, algorithms and models into code and other deliverables through the application of resources. It encompasses the activities of application experts, architects, designers, developers, sales and support personnel, managers, users, etc. Their process related activities are an integral part of the process. All are stakeholders who, in the light of their interests and responsibilities, feed back their perception of needs, expectations, experiences, etc to relevant process agents. Their interactions play a part in driving, directing and constraining the process. All have an impact in and on both process and product.
The fact that the operational system is the product of such global processes imposes numerous challenges when seeking to manage and/or improve either. For example, process changes will have local impact but the global impact may be counterintuitive. Now, with only a few exceptions, e.g., inspections and change management, the emphasis in software process research and practice has focused on the technical process and, in particular, on its forward paths. Feedback mechanisms have, in general, not received much attention despite the fact that they appear to be one of the sources of evolutionary behaviour.
What is known about the evolutionary behaviour of key global process attributes? Investigations over the years have led to the identification of a set of behavioural patterns, termed Laws of Software Evolution, as listed, in their most recent formulation, in Table MML.1 below.
 

No. Brief Name Law
I
1974		 Continuing Change E-type systems must be continually adapted else they become progressively less satisfactory in use
II
1974		 Increasing Complexity As an E-type system is evolved its complexity increases unless work is done to maintain or reduce it
III
1974		 Self Regulation Global E-type system evolution processes are self-regulating
IV
1978		 Conservation of Organisational Stability Average global activity rate in an E-type process tends to remain constant over stages or segments of system evolution
V
1978		 Conservation of Familiarity  In general, the incremental growth and long term growth rate of E-type systems tend to decline
VI
1991		 Continuing Growth The functional capability of E-type systems must be continually increased to maintain user satisfaction over the system lifetime
VII
1996		 Declining Quality Unless rigorously adapted to take into account changes in the operational environment, the quality of E-type systems will appear to be declining
VIII
1996		 Feedback System (Recognised 1971, formulated 1996) E-type evolution processes are multi-level, multi-loop, multi-agent feedback systems

Table. MML.1 : The Laws of Software Evolution (as currently stated)

The laws were numbered in the order of their formulation and have been refined over the years as understanding of the phenomena encapsulated in them has advanced [leh74,85,fea00b]. They were termed laws because they reflect human social, organisational and managerial forces and because they were believed to encapsulate behaviour and phenomena that are, in general, outside the purview of those concerned with day to day development, the direct application of methods, tools and techniques to evolve the software. In general, FEAST results have increased confidence in six of the eight laws. Law VII was not investigated due to lack of appropriate data, though it is supported by theoretical reasoning [leh89,fea00a]. Law IV reflects present understanding and requires further investigation [leh98,raj00]. Based on the laws and on the behaviour they imply, a number of recommendations for long term evolution process planning and management have recently been compiled [leh00]. Readers interested in this and further details regarding the laws are referred to the project web site [fea00a].
The eighth law provides an explicit statement of the feedback system nature of software evolution processes. Its formulation as a law was based on many years of observations of the programming process. As discussed above, the long term growth patterns observed in several systems can be interpreted in terms of compensating positive and negative feedback loops. Positive feedback influences such as pressures to increase functional power to improve sales or augment the customer base, on the other hand, tend to be constrained by the global software process capacity to implement and assimilate change. Given that, in general, managers do not directly or explicitly control system growth rate as such, one may say that observed growth trends are, at least in part, the result of controls imposed by counteracting positive and negative feedback forces.
Additional support for the law has been provided by simulation modelling of software processes. New insight, in particular, has been provided by a number of system dynamics [for61] models built by the FEAST group. These include feedback influences that are believed to be dominant [wer98,cha99,kah00]. Feedback influences tend also to be reflected in simulation-based models of industrial processes presented in the literature, such as those in the ProSim workshops [pro00].
 
 

Role and Impact of Feedback in E-type Processes

As a part of its goals, improvement of the evolution process must seek to achieve systematic and incremental changes to the process and its constituents to obtain gains in productivity, timeliness, product quality and so on. It must also seek to compensate for deterioration of these attributes as a result of, for example, increasing software complexity, loss of key personnel, deterioration of architectural integrity and so on. As mentioned above, the process is in fact a complex feedback system. Difficulties in achieving improvement will, therefore, be experienced if the feedback forces are not adequately considered. Without going into details as to what type of behaviour in the attributes to see of a complex feedback process such as that depicted in figure MML.3 one should expect, one may postulate some immediate observations in the context of process improvement. For example, local process improvements may not be observable from outside the system, that is, they may not be reflected in the overall (global) process or its product. Process changes may have a counterintuitive effect on global process behaviour. The effectiveness of methods, tools and techniques as established in a small setting or in vitro conditions, may not be apparent when such methods, tools and techniques are deployed in a large scale or in vivo situation. Process-local optimisation may not lead to global optimisation and so on. Major process improvement requires, inter alia, appropriate consideration, or adjustment, of sub-processes and feedback-related dynamic influences. Moreover, in general, outer loops that cannot, be strongly, influenced by individual developers, but may have a dominant influence on process outcome. This leads to an overwhelming set of problems whose recognition opens up new approaches to improvement.
 
 

Black-box and White-box Modelling and Models

The starting point of the FEAST investigation was the application of the accepted scientific method (see figure MML.1) to the investigation of the role and impact of feedback in the software process, with particular emphasis on process modelling. It recognised the fact that understanding the dynamic influences in a complex feedback process requires appropriate quantitative models [wid89].
In seeking process understanding, the approach has, to date, involved construction of quantitative models. These aim to reflect the long-term behaviour of attributes of such processes and their product. In so doing, FEAST distinguishes between two types of quantitative models: black-box and white-box. Though different, these are related, even, in some respects, complementary. Black box models encapsulate relationships between attributes. They primarily reflect structure in and behaviour of metrics and hence process inputs and outputs, as sketched in Figure MML.4. The graph formed by solid line rectangles (small) connected by arrows represents a view of a software process, with the rectangles representing, for example, activities that generate or update process documents and other artefacts (including the source code). The arrows represent dependency relationships between such activities. The dashed line shadowed rectangle is intended to suggest that in black-box modelling only behaviour seen from outside the rectangle is of interest. The arrow and the smaller box at the bottom of the figure indicate that the modelling activity follows a top-down approach with successive refinement. To the right of the figure, input-output relationships and input-output behaviour over time or pseudo-time (release sequence numbers RSN [cox66]) and typical relationships of interest are indicated.

Fig. MML.4 : Black-Box Models in FEAST

White box models reflect process structure, its major constituents, sub-processes and relevant details of their interactions. The goal is to convey insight into the internals of the process modelled. Figure MML.5 is analogous to Figure MML.4 but represents white-box modelling activity. The dashed line rectangle, here, encompasses the part of the process being modelled. The right hand side of Fig. MML.5, is intended to suggest that, in principle, any attribute not only inputs and outputs, may be of interest when developing and executing white-box models.

Fig. MML.5 : White-Box Models in FEAST

In the wider software community, the two modelling approaches are generally pursued separately. Examples of black-box modelling as defined here are to be found in the software metrics literature. Activity leading to models analogous to our white-box models has been primarily pursued in the software process simulation modelling community [e.g., pro00]. The FEAST modelling approach and studies, to date have differed in several ways. They have, for example:

The models reflected in the figures that follow illustrate this approach.
The first, Figure MML.6, displays the results of fitting an inverse square model of growth over release sequence numbers (RSN) to data from a number of systems studied in FEAST. The model contains only one parameter E. The prediction error, measured by the mean absolute error (mae) is 8 percent or less, for the systems in the table contained in Fig. MML.6. For two of the systems studied to date, partitioning the interval and fitting separate models over successive intervals improved the fit. Further details on the inverse square models are available in the FEAST literature [tur96,fea00a]. The reader seeking further detail or guidelines in building black box models is referred to [ram00].

Fig. MML.6 : Example of Black-Box Modelling in FEAST

Figure MML.7a presents a system dynamics [for61] model that reflects part of the most recent FEAST/2 work. A detailed discussion of this model and the underlying concepts is given in [kah00]. Further guidelines are given in [ram00].
An example of the output of this model is shown in fig. MML.7b. This figure shows the simulated growth of the system to month 180 compared with actual growth under alternative management policies. These policies relate to

Fig. MML.7a : Example of White-Box Modelling in FEAST

the portion of resources applied to anti regressive [leh74] work such as complexity control. AR = 0.3 indicates that 30 percent of resources are devoted to this activity, AR = 0.0 indicates that no resources are made available to it.

Fig. MML.7b : An Example of the Output of the Model in Fig. MML.7a.



Iterative Modelling for Process Improvement

As a system evolves one must expect to observe the consequences of increasing constraints as the software ages. This may be due to the successive superposition of changes, to the orthogonality of such changes, to increasing remoteness from the original architecture, to growing complexity, personnel turnover and so on. Recognition of such factors provides opportunities for process improvement by investigating and improving appropriate aspects of the process. Such improvement needs to occur over an extended period of time over several releases. Evolution processes tend to be, by their very nature, cyclic with each cycle generating a new release of the operational software. One takes advantage of this by combining black-box and white-box modelling in an iterative fashion [ram00] to exploit the complementary features of the two classes of models to achieve increased understanding of a given evolution process and to support its improvement by means of quantitative models. The diagram of figure MML.8 represents the proposed iterative process. It represents a meta-model for a measurement and modelling driven improvement process. Two basic activities are highlighted: monitoring of key global process attributes by means of black-box modelling and the identification of potential improvements by means of white-box modelling. In principle, models should be updated at every cycle of the iterative evolution process to reflect changes in it.
 
 

Final Remarks

This paper has presented a short summary of the concepts and process modelling approach underpinning the FEAST investigation with the hope that they may be of benefit to both researchers and practitioners in industry and academia. It reports on the work of a single team. It has been clear since the start of the FEAST investigation that achievement of a better understanding of the role and impact of feedback in software processes requires an interdisciplinary approach. Such approach would take into account, for example, business, social, organisational, managerial, cognitive and other aspects of the global process. The investigation would need the co-operative work of many groups. In view of the peculiar characteristics of the software process, a simple transfer of results from related fields (e.g., from management and organisational theory) is unlikely to provide appropriate answers.

Fig. MML.8 : Iterative Modelling

Software processes display and share characteristics with industrial production that transform physical entities. The system dynamics of these have been studied by, for example, Forrester and co-workers since the sixties [for61]. In many respects they also resemble invention, design and R&D processes whose dynamics are not so simple to grasp or to model. Additionally, when compared to other artificial artefacts (in Simon's sense [sim69]), software evolution occurs at a faster rate. Software changes are motivated by organisational and technological changes and frequently triggered and accelerated by the introduction of new releases of the software in the operational domain [leh85]. In this sense, software evolution can be considered the Drosophila Melanogaster, the fruit fly of artificial processes.
 
 

Acknowledgements

We gratefully acknowledge the collaboration of Dr. Paul Wernick for his contribution in the building of the system dynamics model presented and as a member of the FEAST/1 team . Many thanks are due to Ms. Siew F. Lim for her continuing support. Financial support from the UK EPSRC, grant GR/M44101 (FEAST/2) and GR/N02412 (SVFs) is gratefully acknowledged.
 
 

References

references identified with an * are reprinted in [leh85]
 

[bel72]* Belady LA, Lehman MM, An Introduction to Program Growth Dynamics, in Stat. Comp. Perf. Ev., W. Freiburger (ed.), Ac. Press, NY, 1972, pp. 503-511

[cha99] Chatters BW, Lehman MM, Ramil JF, Wernick P, Modelling a Software Evolution Process, ProSim'99, Softw. Process Modelling and Simulation Workshop, Silver Falls, Oregon, 28-30 Jun. 1999, also as Modelling a Long Term Software Evolution Process in J. of Softw. Proc.: Improvement and Practice, 2000, v. 5, iss. 2/3, Jul. 2000, pps. 95-102

[cox66] Cox DR, Lewis PAW, The Statistical Analysis of Series of Events, Methuen, London, 1966

[fea94,95] Preprints of Three FEAST Workshops, Lehman MM (ed.), Dept. of Comp., Imp. Col., 1994/5

[fea00a] FEAST, Feedback, Evolution and Software Technology, web site: http://www-dse.doc.ic.ac.uk/~mml/feast

[fea00b] Preprints of FEAST 2000 International Workshop on Feedback and Evolution in Software and Business Process, Ramil JF (ed.), Dept. of Comp., Imp. Col., London, 10 - 12, Jul. 2000, 124 pp. http://www-dse.doc.ic.ac.uk/~mml/f2000

[for61] Forrester, J., Industrial Dynamics, The MIT Press, 1961.

[kah00] Kahen G, Lehman MM, Ramil JF, Wernick PD, Dynamic Modelling in the Investigation of Policies for E-type Software Evolution, ProSim 2000, 12 - 14 Jul. 2000, London, UK

[leh69]* Lehman MM, The Programming Process, IBM Research Report RC 2722, IBM Research Centre, Yorktown Heights, NY, Sept. 1969

[leh74]* Lehman MM, Programs, Cities, Students, Limits to Growth?, Inaugural Lecture, in Imperial College of Science and Technology Inaugural Lecture Series, Vol. 9, 1970, 1974, pp. 211-229. Also in Programming Methodology, (D. Gries. ed.), Springer Verlag, 1978, pp. 42-62

[leh85] Lehman MM, Belady LA, Program Evolution—Processes of Software Change, Academic Press, London, 1985.

[leh89] Lehman MM, Uncertainty in Computer Application, Comm. of the ACM, Vol. 33, No. 5, May 1990, pp. 584-586

[leh90] Lehman MM, Uncertainty in Computer Application, Technical Letter, Comm. of the ACM, vol. 33, no. 5, pp. 584, May 1990

[leh94] Lehman MM, Feedback in the Software Evolution Process, Keynote Address, CSR Eleventh Annual Workshop on Software Evolution: Models and Metrics. Dublin, 7-9 Sept. 1994, Workshop Proc., Information and Software Technology, sp. is. on Software Maintenance, v. 38, n. 11, 1996, Elsevier, 1996, pp. 681 - 686

[leh98] Lehman MM, Perry DE and Ramil JF, On Evidence Supporting the FEAST Hypothesis and the Laws of Software Evolution, Proc. Metrics'98, Bethesda, MD, 20-21 Nov. 1998, pp. 84-88

[leh00] Lehman MM, Rules and Tools for Software Evolution Planning and Management, FEAST 2000 Workshop, 10 - 12 July 2000, Imp. Col.

[pro00] Prosim 98, 99 and 2000, Software Process Simulation and Modeling Workshops, http://www.prosim.org

[ram00] Ramil JF, Lehman MM, Kahen G, The FEAST Approach to Quantitative Process Modelling of Software Evolution Processes, Proc. PROFES'2000 2nd International Conf. on Product Focused Software Process Improvement, Oulu, Finland, 20-22 Jun. 2000, in Frank Bomarius and Markku Oivo (eds.) LNCS 1840, Springer, Berlin, 2000, pp. 311-325

[raj00] Rajlich VT and Bennet KH, A Staged Model for the Software Life Cycle, Computer, July 2000, pp. 66 - 71

[sim96] Simon HA, The Sciences of the Artificial, 3rd. ed. The MIT Press, Cambridge, MA, 1996, 231 pp, first pub. in 1969

[tur96] Turski WM, A Reference Model for the Smooth Growth of Software Systems, IEEE Trans. Softw. Eng., v. 22, n. 8, Aug. 1996, pp. 599 - 600

[wer98] Wernick P and Lehman MM, Software Process White Box Modelling for FEAST/1, ProSim '98 Workshop, Silver Falls, OR, 23 Jun. 1998. As a revised version in J. of Sys. and Softw., v. 46, n. 2/3, 15 Apr. 1999

[wid89] Widman LE and Loparo KA, Artificial Intelligence, Simulation, and Modeling - A Critical Survey, in Widman LE, Loparo KA and Nielsen NR (eds.), Artificial Intelligence, Simulation and Modeling, Wiley, NY, 1989

[zur67] Zurcher FW and Randell B, Iterative Multi-Level Modelling - A Methodology for Computer System Design, IBM Res. Rep. RC 1938, Nov. 1967, IBM Res. Centre, Yorktown Heights, NY 10594. Also in Information Processing 67, Proc. IFIP Congr. 1968, Edinburgh, Aug. 1968, pp. D138 - 142
 


 
Partners in EuroSPI

Editors
ISCN LTD, ISCN GesmbH, Schieszstattgasse 4/24, 8010 Graz, and Coordination Office, Florence House, 1 Florence Villas, Bray, Ireland, office@iscn.at, office@iscn.com, office@iscn.ie, Editing Done: 19.7.2002