Key Success Factors for Business Based Improvement
Miklos Biro Sztaki, Budapest, Hungary Richard Messnarz ISCN, Dublin, Ireland & ISCN Regionalstelle, Graz, Austria
This paper is based on the results from the EU Leonardo da Vinci project PICO and discusses software process improvement from the business manager's viewpoint. Thirty experts across Europe with different backgrounds (process analysis, goal analysis, measurement, experimental approaches, business analysts) from ESPRIT projects running in the 90's have contributed to the work of PICO [3]. The result is a framework into which all methodologies fit describing how they can be combined to achieve the business manager's goals and perceptions. Experiences and case studies have been contributed from Alcatel, Siemens, Festo, and many smaller and medium sized firms from 11 different EU countries. Specifically the paper discusses key topics to be taken into consideration : business strategies, process and business indicators and their relationship, goal analysis and measurement, people factors, and infrastructure issues. The establishment of a quantitative feedback and learning cycle is emphasized.
The PICO Book:
R. Messnarz, C. Tully, (eds.), Better Software Practice for Business Benefit, IEEE Computer Society Press, Tokyo, Brussels, Washington, November 1999, ISBN : 0-7695-0049-8. It has been written by 30 authors from 11 EU countries.
There are many definitions of quality which are extensively discussed in textbooks. Here we only mention two of them to show the slight difference between them in technical vs. business orientation.
Quality. totality of characteristics of an entity that bear on its ability to satisfy stated or implied needs. [ISO 8402:1994, adopted in ISO/IEC 9126]
Quality. The degree to which a system, component, process, or service meets customer or user needs or expectations. [IEEE-Software Engineering Standards]
It was also recognised that in the software development process, there are several stakeholders who have naturally different views of quality. Three views are identified in the ISO/IEC 9126:1991 standard. Even though these views are incorporated in a more comprehensive quality model in the currently draft version of the new ISO/IEC 9126 standard, we prefer to present the original definitions below.
"User's view... Users are mainly interested in using the software, its performance and the effects of using the software. Users evaluate the software without knowing the internal aspects of the software, or how the software is developed...."
"Developer’s view. The process of development requires the user and the developer to use the same software quality characteristics, since they apply to requirements and acceptance. When developing off-the-shelf software, the implied needs must be reflected in the quality requirement...."
"Manager's view. A manager may be more interested in the overall quality rather than in a specific quality characteristic, and for this reason will need to assign weights, reflecting business requirements, to the individual characteristics.
The manager may also need to balance the quality improvement with management criteria such as schedule delay or cost overrun, because he wishes to optimise quality within limited cost, human resources and time-frame."
[Excerpts from the text of the ISO/IEC 9126:1991 standard on Information technology - Software product evaluation - Quality characteristics and guidelines for their use.]
Our objective in this paper is to elaborate on the last one of the above views.
A business manager sees process improvement or the achievement of quality under a different perspective. He wants to generate business, secure markets, increase shares, and get new contracts. Thus his interest is to create economic feedback loops that allow an increase of business potentials by investment into process improvement.
Figure 1 : A Process – Business Feedback Loop
As it is illustrated in Figure 1 investment is possible on single process factors or a combination of them, and the change in the actual design/production/delivery process will impact a set of business factors.
The business manager’s greatest burden to solve is to find a traceable feedback relationship between process factors and business performance factors which will pay back.
Another important aspect is that the market (in a business environment) sometimes does not decide by actually measured quality, sometimes the perceived quality of the customer counts.
An example is the strategy of some Japanese firms to enter the European market, as it is perceived by the European competitors. A European firm (a radio manufacturer, also developing the software for radios) always was confirmed to produce radios which are running through extensive tests and thus have to be sold at a certain minimum price. The Japanese competitor realized that there is a market demand for different radios (at different price and quality levels) and started to develop and distribute radios also at low cost (with lower quality and functionality). However, not the quality of the system but the perceived quality from the different levels of the customers was deciding the market success, so that many customers bought the low price radios and the European radio sales became smaller. This does not mean that the Japanese firm was not offering quality, it rather means that they offered different systems with different quality levels (of course, also including systems of the same high standard than the one offered by the European firm).
This then comes to new success factors like flexibility, and configurability which have to be combined with the standardisation approaches underlying the process improvement models.
And in a typical business scenario thus a division could produce 100% quality and achieve high organisational maturity grades, but it still could fail in business due, for instance, missing flexibility and system configurability.
Another example is the Microsoft and Apple story. A major reason why the quality of a MaIntosh is perceived (by the customers) as higher than that of a PC is the fact that a Mac (once bought) kept stable over years, whereas due to a upgrade philosophy of rapidly changing versions with exponentially increasing resource demands the PCs are just stable for about 1.5 years and then have either to be largely upgraded or exchanged.
Now imagine a situation in which the same game starts with the Mac, where due to largely increasing resource demands also the Mac has to be upgraded every 1.5 years. This would lead to a loose of the stability argument and the customers would perceive the quality of PC and Mac as the same, thus buying the low cost PCs in the future.
Of course, this is just a story, but it shows some ideas of typical business strategies, which at the end calculates not only with defect rates but with perceived quality from customers.
Business Manager’s Quality Perception. A business manager invests into processes, people, and infrastructure with the aim to satisfy market demands and perceived quality from customers, with a view of creating a traceable process – business feedback loop.
Below, we present an original Structured View of Business Motivations for SPI first published in the PICO book.
One of the most pertinent questions a business manager can ask is the following "How can I make my firm succeed where another fails?" Managers with financial, operating, production, marketing, human behavioural, or other orientations will give a variety of answers to this question and will arduously argue for their valuable ideas. Here, we will outline a framework integrating and structuring several orientations [4], [5], [6], [19].
The key concept of the approach is the notion of lever. Levers are means used by a firm to increase its resource generating ability, just as a mechanical lever is used for increasing the force applied to an object. The analogy goes even further. Just as a force can be applied in many different ways to the object resulting in a similar displacement, the use of the different levers can increase the resource generating ability of the firm resulting in similar business benefits. Finally, the resources are used to increase the assets of the firm and to reward employees and stockholders.
Let us analyze the ways software process improvement can provide leverage to a firm from the financial, operating, production, marketing, and human behavioural perspectives.
Financial leverage means borrowing funds and investing them with a return higher than the cost of the debt. If a company is able to exploit financial leverage, it can make money on funds it does not own. Can software process improvement provide financial leverage to a firm? The answer is clearly yes if the return is high enough to make it worth borrowing money for achieving it. In other words, is the ROI (return on investment) for software process improvement high enough?
This issue is discussed in more detail in the PICO book. Here, we only mention a few determining numbers which allow the reader to form an idea about the magnitude of the leverage that can be achieved. ROI is considered as "the bottom-line figure of most interest to many practitioners and managers" in a pioneering report [14,15] of the Software Engineering Institute of Carnegie Mellon University. The value of this report lies in the fact that it contributes to the satisfaction of the major need of companies for quantitative information regarding the benefits of SPI before committing resources and investing into it. There were 13 organisations, where CMM® -based SPI [17] occurred prior to 1990, which agreed to provide their highly sensitive and confidential data to the SEI for anonymous or identified reporting according to their own decision. The companies were the following:
Bull HN
GTE Government Systems
Hewlett Packard
Hughes Aircraft Co.
Loral Federal Systems (formerly IBM Federal Systems Company)
Lockheed Sanders
Motorola
Northrop
Schlumberger
Siemens Stromberg-Carlson
Texas Instruments
United States Air Force Oklahoma City Air Logistics Centre
United States Navy Fleet Combat Direction Systems Support Activity
The report does not claim that its results are typical. It only shows the potential benefits of SPI in a favourable environment. And these benefits in terms of ROI are more than impressive. The range of the ratio of measured benefits to measured costs is between 4x and 8.8x over periods of software process improvement ranging from 3.5 to 6 years. Benefits include savings from productivity gains and fewer defects, but do not include the value of enhanced competitive position which will be examined below under the title marketing leverage. Costs include the cost of a Software Engineering Process Group (SEPG), assessments, and training, but do not include indirect costs such as incidental staff time to put new procedures into place.
Capers Jones [9] reports an ROI of 3x to 30x with the returns measured over a 48 months period using the Software Productivity Research (SPR) assessment method and baseline studies. Exceptionally in the literature, Capers Jones, [9] also reports a negative and alarming record: "Several companies and government agencies have managed to spend in excess of $10,000 per capita with no tangible benefits accruing."
The conclusion of this section is that yes, software process improvement can provide significant financial leverage to the firm making it worth borrowing money for investing into it. Nevertheless, the effect can be adverse if the company does not pay appropriate attention to accrued costs and to immediate exploitation.
The profitability of a firm highly depends on its cost structure, that is the repartition between its fixed costs and variable costs. Operating leverage means the relative change in profit induced by a relative change in volume, which is clearly higher for a firm with lower variable costs. Nevertheless, the achievement of a low variable cost production usually presumes high fixed costs, that is a capital intensive process.
Figure 3: Firm with low fixed costs (FC) and high variable costs (VC). The total cost is TC = FC + VC.
Software process improvement clearly means an increase in fixed costs, which include training, consulting fees, equipment, software licenses and improvements in office conditions. However, the question is whether the company is really able to use it for decreasing its variable costs. Measuring the variable costs of software production is not a straightforward issue. The notion of function point had to be invented to resolve this problem among others. Function point analysis is discussed in more detail in chapter 6 of the PICO book.
Function points are results of well defined calculations based on different characteristics of a software product that are of interest to users: inputs, outputs, data groups, inquiries, interfaces. The cost of the average number of person months necessary for delivering a fixed quantity of function points is a sample measurement for the variable costs of software production (person months/function point). However, the measurement mostly used in reports on software process improvement is development productivity (function points/person month) which is in fact the reciprocal of the above number. In the following, reported results are presented in terms of development productivity whose increase is consequently equivalent to the decrease of the variable cost of software production.
If, due to software process improvement, a software firm is able to deliver the same quantity of function points using less person months than its competitors, then it will have the potential to take advantage of operating leverage. Nevertheless, real profit will only be generated if the revenue resulting from actually delivered function points exceeds the total of the cost (TC) of software process improvement (FC) and the cost of person months used for generating them (VC). This means that bigger firms with a larger number of delivered function points will have a better chance to enjoy the operating leverage resulting from software process improvement.
Productivity gains per year measured in lines of code (LOC) per unit of time are reported in [14,15] to be between 9% and 67% at the examined organisations. Another form of productivity gain particularly relevant to software is due to the earlier detection of defects also presented in [14,15]. The figures show a 6-25% increase in the number of early detected defects. This represents enormous savings if we consider that "the cost of fixing a defect pre-release is approximately $50 per line of code, while the cost of fixing a defect discovered post-release is about $4000 per line of code".
It is important to highlight at this point that the major European company SIEMENS is a world-wide pioneer in measuring and publishing information related to productivity gains resulting from software process improvement. They report the following experimental reductions in error costs based on maturity levels [25]:
17% from Level 1 to Level 2,
22% from Level 2 to Level 3,
19% from Level 3 to Level 4,
44% from Level 4 to Level 5.
SIEMENS also reports productivity increases in terms of lines of code, but at this point it is more appropriate referring to chapter 8 of the PICO book giving direct account of SIEMENS experiences.
Another report accounting for productivity gains due to process improvement in European software development organisations originates from the results of a questionnaire developed by IBM Europe [13]. This report compares, among others, the performance of leaders with that of laggards from among 360 responding organisations from 15 countries. While leaders "achieve a development productivity of 25 function points per person month; remove over 95% of defects before delivery; estimate consistently to within 10% of the actual cost and duration of a project; and spend less than 1% of the development effort on defect correction in the first 12 months after delivery", laggards "have a development productivity below 5 function points per person months; remove less than 50% of defects before delivery; have projects which often exceed estimates by more than 40%; and spend more than 10% of the development effort on defect correction in the first 12 months after delivery".
Production leverage is the rate of growth of profits resulting from cost declines due to the accumulation of production. It is an empirical fact that unit production costs decline exponentially when experiences are accumulated and the steady reuse of these experiences is well managed by the firm.
Figure 4 : The Experience Curve in a logarithmically scaled system of coordinates
The graph of the unit costs in function of the cumulative quantity produced is called the experience curve which is usually represented as a straight line in a logarithmically scaled system of coordinates. The existence of the experience curve is essentially due to economies of scale, learning, improvements, and reuse.
The accumulation of experiences and the management of their steady reuse is clearly one of the primary objectives of software process improvement. Interestingly, this aspect of software process improvement has not been analysed directly. Nevertheless, the paper [14,15] acknowledges that the techniques useful for tracking the cost changes over time do not specify what is causing the changes. The cause may be the process improvement, but it may also be increased experience, new analysis and design methods, new tools, and so on.
Marketing leverage means the effect of higher prices and innovative distribution on profits. Software process improvement, maturity achievement, ISO 9000, or TickIT certification have an important impact on the perceived capability of the company and on the perceived value of its products, which contributes to improved customer satisfaction and makes it possible to achieve higher prices.
Quality and process improvement are part of a differentiation strategy in which the business delivers and is perceived to deliver a product or service superior to that of competitors. In a study of 248 distinct businesses in the service and high-tech industries referred to in [1], "a reputation for quality was the most frequently mentioned sustainable competitive advantage".
In line with the above US study, a major European company, Lloyds Bank Plc. lists the demonstration of competitiveness through CMM/SPICE/TICKIT certification as one of the key drivers for software process improvement [18].
The experiences of SIEMENS, another major European company, were already mentioned and are described in more detail in chapter 8 of the PICO book. The "promotion of the external visibility of Siemens’ software competence" is listed as an important area to focus [25]. They also report that "highly-predictable quality regarding system releases and costs led to greater market acceptance".
[14,15] acknowledge the importance of "improved reputation, good will, and brand name recognition" as intangible benefits of process improvement arising from the "impact of SPI on customers", but they present no actual results relating to these.
A survey reported in [10] provides the feedback of more than 50 companies on the benefits gained from the TickIT certification scheme. One of the major benefits is formulated in the following way: "Customers have increased confidence in the quality of our products. With the advent of TickIT the UK Ministry of Defence (and many other influential purchasers) have ceased their second party assessment activity, while many other large organisations now insist that their suppliers and product resale partners achieve ISO 9001 certification by a TickIT accredited certification body."
One of the rare reports which provide a measurement of the direct effect of software process improvement on the marketing leverage of a company was presented by Peterson [24]. The report is based on 560 SEI software process assessments through December 1995 whose results were provided to the SEI by March 1996. The statistics give the percent of respondents reporting "excellent" or "good" customer satisfaction when improving software processes from the initial level (around 80%) through the repeatable level (a surprising decrease to 70%) upto the defined level (around 100%).
Human leverage means the effect of employee motivation on profits. It is widely known that employee motivation (empowerment) can be significantly influenced by immaterial means like management styles and organisational structures. Huge individual energies can be released for example in an appropriate teamwork environment where team members are simply given the responsibility to do their jobs as well as they can, instead of exerting close surveillance over them. Nevertheless, attention must be paid at the differences in the collective mental programming of people in different national cultures [12].
The exploitation of human leverage is particularly important in software process improvement since software development is a fundamentally human mental process.
Herbsleb & al. [14,15] classify the "impact of the SPI effort on the organisation’s employees", including "better morale, improved understanding of the corporate mission and vision, fewer crises, less stress, less turnover, and better communication within the organisation", among the intangible benefits of SPI. Actually, no measurements are presented relating to these benefits in this report.
The already mentioned report presented by Peterson [24] also shows statistics giving the percent of respondents reporting "excellent" or "good" staff morale when improving software processes from the initial level (around 20%) through the repeatable level (around 50%) up to the defined level (around 60%).
There is an already mentioned important study, [13] initiated by IBM Europe, which gives a measurement of the impact of employee morale on the level of performance of a company. The statistics based on 360 responding organisations from 15 European countries show that employee morale correlates strongly with both delivery performance and quality performance levels.
An ultimate recognition of the importance of human leverage is the development of models directly addressing this issue: the People Capability Maturity Model (P-CMM) [11], and the Personal Software Process (PSP).
When PICO started in 1995 each of the different methodologies argued to be the best, to insert the most important rules and principles. But over the project time and comparing different industrial case studies it turned out that you are most efficient if you take into account all principles from all methodologies.
It is equally important (as stated in [19])
So this electronics company decided to run a goal analysis (based on the GQM approach) in parallel interviewing business managers, department heads, IT managers and project managers and designing a consistent goal tree from top to bottom.
One of the specific business policies was to create a financial framework for the next years which allows to get a reserve budget to fight for a brand new product on the market. To get to this marketing budget the business managers decided to stabilize the development effort from divisions at x% so that with all other overheads and cost a certain percentage is saved every year to have the budget together right at the time when the product is announced. At this moment the divisions were certainly higher than x% and the improvement actions (based on the previously identified weaknesses from the assessment) had to demonstrate after 3 years the success of achieving this business goal.
The technical staff were frightened and thought that people will be dismissed but the truth was that a proper interpretation of the business goals led to a completely different view. The business managers expected that process improvement provides a better work process and environment so that with the same staff more projects and tasks can be done and over time the development effort is stabilized at x%.
Under this perspective the 3 process weaknesses were again analyzed and further interviews showed a potential of re-use because in all systems in the sector nearly 80% of the functionality was always the same. So the improvement plan focused on an integration of design, configuration management and review of a re-use pool of these 80% functions and to reduce the development for each project to the only 20% additions, thus enhancing productivity and achieving the effort stabilization.
Now, let us assume that only a pragmatic assessment would have been performed. Three weaknesses would have been identified and without re-use orientation would have led to a pragmatic proposal to first run a pilot project to identify a configuration management system and field test it, to disseminate it to other projects, and to help making it a division wide standard. Sounds simple, BUT unfortunately the business context is lost. What then happens is that management sees additional effort, the development effort further increases, and with no vision of decrease of the development effort the business manager after 1 year (before benefits can be made visible) would really decide about things like dismissals.
This example refers back to the principles of operating and production leverage.
Business managers, project managers, and practitioners speak different languages and might have different viewpoints on the same situation [22,23] . Business managers speak about fixed cost, variable cost, return on investment, leveraging, market trends, product sales, and customer satisfaction. Middle and project managers speak about budget, work plans, quality plans, configuration management, requirements analysis and structured analysis, and always fear to be delayed or to overrun the budget provided by the business managers. Practitioners deal with modules, design them, implement and test and deliver them so that they can be integrated into the system architecture planned by the project manager.
It is the nature of process improvement methodologies that measurement and control functions are installed which again will be seen differently from the different target groups. Business managers not understanding that SPI needs investment with a ROI in about 3 years sometimes demand that process improvement is performed without any assignment of budget to it: lets do quality but it should not cost any dollar. This certainly leads to a disaster and top management commitment is the number one success criteria for starting an improvement program. Middle managers will like the process improvement most because it provides them with methodologies and facilities to better define the processes, to better visualize the productivity and quality, and to improve the predictability which leads to the fact that schedules and budget are kept satisfying therefore the business managers. At the beginning the practitioners usually see the implementation of a process improvement program as a dirty trick of middle and project management to better control their performance.
However, after some time they start to realize that more reliable plans give them enough time for design, better design reduces the re-work and maintenance stress. Formalized reports help them to identify the root cause of problems and to track the correction, and they can learn and improve themselves based on measures [ 22,23] .
It is a key to success to have all groups behind the initiative and to act as a translator of the different viewpoints.
For instance, in a European project Bestregit (which applied PICO principles in general management and technology transfer institutions) role based team-work models were established for best practices, and an experiment at a Spanish site showed that using these role based models leads to
For instance, PICO principles were applied for ISO 9001 certification based on computer supported team-work scenarios (tried out with certifications at TÜV, ÖQS, and Norske Veritas). Measurements at Austrian sites showed that the actual motivation of engineers to use the standards grew from 17% to 57% of the staff.
These effects relate back to the principle of human leverage.
On an Intranet Scale –
A major fear of engineers when new standards are introduced is the additional expected amount of documentation.
In old traditional situations of standard introduction a process group is established which creates and maintains the standard, produces a manual, and looks that all projects keep the guidelines described in the manual.
However, engineers do not like to see quality or improvement as a separate part in parallel to their normal engineering work. Quality and improvement related processes should be an integrated part of their work place, just like a compiler is.
This leads to Intranet based systems [22] that support
European projects have tested such environments in companies and found, for instance [5], a number of key factors:
On a Global Scale -
To stay competitive on the global market it is necessary to set up win-win based agreements in cost sharing projects in which partners from different countries share the risk and the effort and jointly exploit ideas, products, and services. Through effective and distributed collaborations organisations can cut down their risk significantly (e.g. sharing the development cost with other partners) and can reach a much larger market [22].
However, the key problem is that distributed collaboration needs effective co-ordination of the work of the different partners. And old conservative means such as direct supervision, local meetings, large local and not distributed teams, do not work any more. The decomposition into smaller competence teams with clear cooperation interfaces supported by new and effective communication systems is needed. This includes a virtual office on the net with project archives and document management, configuration management, guide-lines and computer support for project documentation, network and computer supported information flows, and appropriate security mechanisms assuring privacy of the materials exchanged and produced.
A field test implementation of such a system at different companies is described in [22]. Companies using the system described in [22] have successfully achieved ISO 9001 certification based on fully computer supported team-work processes.
This new approach of collaborative development leads to a big chance for creating financial leverage (by joint risk and effort funding) and an increased marketing leverage (by joint representation on the market, and larger distribution through a net of partnerships).
SPI [19] is a strategy for business managers to align their business goals with technical improvement objectives, to apply a set of different methodologies
to create a process – business feedback loop, with the aim to make the organization more competitive (Marketing Leverage). Business orientation is a must to create the budget for SPI (Financial Leverage). People management and motivation is a must to get a critical mass of people following the SPI vision (Human Leverage). Goal trees are a must to translate the business manager's viewpoints into practical objectives for the SPI teams (aiming at Production and Operating Leverage). And pragmatic assessment methods (CMM, Bootstrap, TickIT, Trillium, etc.) are just one tool to evaluate the strengths and weaknesses before applying the combination of the other approaches (from business orientation to goal analysis).
PICO produced a book [19], and a set of training courses for
PICO also developed a tool set which can work (by configuration) with almost all different assessment methodologies. This was required because PICO contributors came from many different methodology backgrounds. PICO did not develop a new methodology, it is rather a collection of experiences of how to combine existing approaches to achieve a business based strategy. PICO’s book (although announced earlier already) is still at IEEE in the production, and when the book comes out PICO will distribute at moderate prices a CD with all courses, the tool, and further SPI materials as a complementary set to the book.
[1] Aaker,D.A. Strategic Market Management. John Wiley & Sons, Inc., 1995. [2] The ami Handbook, 1995, ISBN 0-201-87746-5 [3] Biró,M.; Feuer,É.; Haase,V.; Koch,G.R.; Kugler,H.J.; Messnarz,R.; Remzsö,T. BOOTSTRAP and ISCN a current look at the European Software Quality Network. In: The Challenge of Networking: Connecting Equipment, Humans, Institutions (ed. by D. Sima, G. Haring). (R.Oldenbourg, Wien, München, 1993) pp.97-106. [4] Biró,M.; Sz.Turchányi,P. Systems of Decision Criteria Supporting Software Process Improvement. In: DSS - Galore, Proceedings of the fifth Meeting of the EURO Working Group on Decision Support Systems (ed. by M. Brännback, T. Leino). (Institute for Advanced Management Systems Research, Turku, Finland, 1995) pp.133-147. [5] Biró,M.; Feuer,É.; Remzsö,T.; Sz.Turchányi,P. Business Decision Problems Supported by Software Product and Process Assessment. In: Proceedings of the ESI-ISCN '95 Conference on Practical Improvement of Software Processes and Products (ed. by T. Katsoulakos, R. Messnarz). (European Software Institute-International Software Consulting Network, Vienna, Austria, 1995). [6] Biró,M.; Sz.Turchányi,P. Software Process Assessment and Improvement from a Decision Making Perspective. ERCIM News (European Research Consortium for Informatics and Mathematics) No.23 (1995) pp.11-12. (http://www-ercim.inria.fr/www-ercim.inria.fr/publication/Ercim_News/enw23/sq-sztaki.html) [7] Biró,M.; Feuer,É; Ivanyos,J. Process Improvement Expriment at MemoLuX. In: Proceedings of the ESI&ISCN 1997 Conference on Practical Improvement of Software Processes and Products (ed. by R.Messnarz). (International Software Collaborative Network, Budapest, 1997) pp.6.18-6.29. [8] Biró,M.; Remzs?,T. Business Motivations for Software Process Improvement. ERCIM News (European Research Consortium for Informatics and Mathematics) No.32 (1998) pp.40-41. (http://www-ercim.inria.fr/www-ercim.inria.fr/publication/Ercim_News/enw32/biro.html) [9] Capers Jones, The Pragmatics of Software Process Improvement. Software Process Newsletter. No.5, Winter 1996, pp.1-4. [10] CSA, TickIT provides proven quality benefits to both customers and suppliers of software systems. CSA (Computer Services Association) Position Paper. Pub. No. 32, 1994. [11] Curtis,B; Hefley,W.E.; Miller,S. Overview of the People Capability Maturity Model. Software Engineering Institute, Carnegie Mellon University, Maturity Model CMU/SEI-95-MM-01. [12] Geert Hofstede, Motivation, Leadership, and Organisation: Do American Theories Apply Abroad? Organisational Dynamics. Summer 1980, pp.42-63. [13] Goodhew,P. Achieving real improvements in performance from software process improvement initiatives. European Software Engineering Process Group Conference 1996 (C306). [14] Herbsleb,Jim; Hayes,Will. Performance Profile: Measuring the Business Value of Software Process and Technology Improvements. European Software Engineering Process Group Conference 1996 (C317). [15] Herbsleb,J; Carleton,A; Rozum,J; Siegel,J; Zubrow,D. Benefits of CMM-Based Software Process Improvement: Initial Results. Software Engineering Institute, Carnegie Mellon University, Technical Report CMU/SEI-94-TR-13. [16] Ivanyos,J.; Biró,M.; Messnarz,R. The PASS Process Improvement Experiment in Hungary (PASS EP 21223). In: Proceedings of the EuroSPI'1998 Conference on How to reap the business benefit from Software Process Improvement (ed. by R.Messnarz). (International Software Collaborative Network, Gothenburg, 1998) pp.8.24-8.45. [17] Key Practices of the Capability Maturity Model V 1.1, CMU/SEI-93-TR-25, Feb. 93 [18] Larner,C. More practical experiences and lessons gained by the software engineering process group of a major European bank: a question of ownership. European Software Engineering Process Group Conference 1996 (C304). [19] R. Messnarz, C. Tully, (eds.), Better Software Practice for Business Benefit, IEEE Computer Society Press, Tokyo, Brussels, Washington, ISBN : 0-7695-0049-8, November 1999 [20] Messnarz R., et. al. , BOOTSTRAP: Fine Tuning Process Assessment, IEEE Software, pp. 25-35, July 1994 [21] Messnarz R., Kuvaja P., Practical Experience with the Establishment of Improvement Plans, in: Proceedings of the ISCN'96/SP'96 Congress on December 1996 in Brighton, pp. 155-169, ISCN Ltd. Dublin, Ireland [22] Messnarz R., et.al., NQA – Network based Quality Assurance, in: Proceedings of the 6th European Conference on Quality Assurance, 10-12 April 1999, Vienna, Austria [23] Thamhain H.J., Wilemon D.L., Criteria for Controlling Projects According to Plan, in (eds.) Nahouraii E., Butler J.T., Petry F.E., Richter C., Tham K., Software Engineering Project Management, pp. 15-54, IEEE Computer Society Press, USA 1990 [24] Peterson, B. Software Process Improvement Trends. European Software Engineering Process Group Conference 1996 (C306). [25] Völker, A; Gonauser,M. Why Maturity Matters. European Software Engineering Process Group Conference 1996 (C305).