Personal Software Process implementation
In a production environment
Stavros K Menegos, MSc Computer Logic SA, Senior S/W Architect Dr. Charalampos Avratoglou Computer Logic SA, S/W R&D Director
PERSPI is an experiment that aims at improving the way individuals perform their day-to-day activities in the context of software development. More specifically, PERSPI is an attempt to introduce the PSP methodology in an industrial environment, putting emphasis on its gradual employment in a real-life project, rather than on an introduction through formal and long-term training.
The presentation gives an overview of the environment on which the experiment is applied, i.e. Company’s business area and strategy, the objectives of the experiment and the baseline process characteristics. Next the PIE’s structure is given and the findings and lessons-learned are presented. Finally future actions and areas of investigation are discussed.
The above efforts where formed to support the business strategy for a development process based on State-of-the art technology, Software Reuse and Process Repeatability.
The PERSPI PIE has been designed to complement the above efforts, addressing the finest instrument of the software development process, the individual developer.
In the software-engineering field a comprehensive OMT-based object-oriented methodology is currently used. A set of tools are used including an upper CASE tool, requirements-features-development items tracking database, a version control tool and an effort tracking and measurement tool.
The language is C++ and visual programming tools are used for the user-interface parts. Productivity is greatly enhanced by an in-house developed repository-based, OO-aware, dedicated application framework (the OMEGA framework) which encapsulates the generic application architecture and provides a set of reusable components. The framework also encapsulates a host of standardised quality rules that apply to the user and to the application developer interface.
The Organisational and Process infrastructure has been reorganised according to the CMM model and has been assessed as a level-2 with traces of level-3 process. Additionally the company is ISO-9001 certified for the development, maintenance and support of S/W.
The above top-down approach to process improvement, while promising and essential, seems to have low penetration to the micro-processes of individual developers, while it presents a significant cost for managerial overhead. On the other hand, the size of the development department and the traditional reliance on individual developers for quality and efficiency justify the need of improving the individual developer’s process, and this idea has immediately gained management commitment.
It was anticipated that the successful introduction of Personal Software Process principles would improve both product quality, development efficiency and productivity by introducing best practices at the finest level of the development process, the individual developer.
It was also expected that this approach would directly involve developers in process improvement and would facilitate the institutionalisation of several improvement efforts done in the past in the software engineering, the quality assurance and the process management fields.
The following were expected at the end of the PERSPI project:
The reference points and desired outcome determining the success of the experiment for the experiment have been set:
The next step was to select the developers that would form the PSP class and participate in this distance learning course. We selected five Senior S/W Engineers that were involved in the development of core S/W Omega projects. In this team, the Omega Framework S/W Architect was also included. This role is responsible for the Analysis, Design and Implementation of the key features of the Omega library Framework on top of which all the Omega applications are implemented. All of the engineers had a very good experience in C++ programming language and in the development of high-quality S/W products under strict deadlines. They were also heavily involved in the previous Process Improvement efforts, mainly the CMM Process Improvement Project and the ISO 9000-1 certification. That means, that the necessity for the process improvement in the finest level of the developer was realised and very well understood by the whole team.
The participation in the PSP class required a significant amount of time and effort for the successful completion of the course. The estimated time was twenty hours per week per developer but the actual effort averaged to twenty-five hours. However, the training took place during a very demanding period, which did not allow for proper buffering of the developers from their projects. Thus commitment and self-sacrifice was necessary for the success of the training.
One of the advantages of having the education program in parallel to the work was that even from the first lectures on the PSP the developers applied certain aspects and guidelines in their everyday S/W development practice. This was a promising issue for the success of introduction of the PSP in our production environment.
A very important criterion for the construction of the pilot PSP team was the ability of the team members to train the whole Omega S/W development community to the PSP in the context of the existing practices and needs. For this purpose, we assembled a second team of five S/W Engineers who were recently introduced in the Omega S/W development process. Two members of the first pilot team trained this second team on the PSP. The education program for the second team started four weeks after the commencement of the PSP course. The reasons for which we established this second team were:
As far as it concerns the training of the second team, this was a more difficult task to complete. The main problem was that the commitment of the participants to this effort was not as high as necessary. The result was that only two of the five developers actually completed the course. The second problem was that the instructors (two developers from the pilot team) did not have the available time to thoroughly examine the submitted lab work. However, as explained in the previous paragraphs this training effort gave us better insight on the possible implications we would meet when introducing PSP in junior members of the Omega development process. Actually we reached to the conclusion that the PSP training should be combined with the education – training of the new comers in Omega processes and Infrastructure. Another interesting conclusion is that having the PSP training performed by external institutes through distant learning is a better motivation for the participants for they are individually awarded.
The main characteristic of the Omega S/W development process is the ability to quickly address the diverse needs of the enterprises and to develop competitive products for a wide range of business domains. The development process is assisted and supported to large extent by our custom developed library framework on top of which applications are developed.
The PSP as a means for improving predictability, productivity and quality is very promising. However, introducing and integrating the PSP as described in theory our development process and framework has certain advantages and disadvantages. The cons and pros are summarised as following:
Cons
The productivity of the MIS Reports development process was poor and the quality of the delivered systems was rather low. In order to improve this process we performed two combined actions. The first one was to improve the technological infrastructure (Omega Development platform) and to provide the Report developers with a powerful platform that encapsulates the difficulties of Report writing and facilitates this process. The second action was to experimentally introduce CL_PSPR1 into the Report development process.
The CL_PSPR1 process is a modified version of CL_PSP1 that has been tailored and customised to address the needs and the individual characteristics of Report development process. This was a rather interesting experiment with very promising results. First of all we had to provide the semantics of the mapping of the S/W Code world to the Report development world. For example the concept of object was mapped to the concept of the Report or Report Element (Cross-tab report, Section Element, Group Element, etc.). The next step was to provide the appropriate categorisation of Report Elements and their estimated effort based on the data we gathered from the Report Development process. The categorisation of Report Elements is described in Appendix A: Table SME.2.
Furthermore, we introduced the following Defect Type standard for the defect categorisation in the Report Development process as described in Table SME.3.
The action of compilation was mapped to the action of Preview – Run of the Report. Testing remains the same in principle with the CLP_PSP1 (we used the same Test Templates and test forms as in CL_PSP1) since the semantics of testing is to validate that the delivered item performs correctly according to the specifications. Regarding the reviews and the checklists, CL_PSPR1 has also a Design Review and a Design Checklist while instead of Code review it defines a Report Layout Review (Report Layout checklist) and a Database Processing Review (Database Processing checklist).
The report developers were not formally trained to PSP but only to CL_PSPR1 process and to the significance of the introduced metrics. Since CL_PSPR1 is rather a simple and straightforward process with minimal overhead no significant resistance has been identified. On the contrary, the report developers showed enthusiasm and commitment to fill-in and take advantage of the collected data.
The raw data from enacting CL_PSP1 were processed and interpreted and we reached to the following results:
PSP has been proven to be a quite effective approach in improving the development process from CMM Level 3 to Level 4. There is a significant gap between these levels that has mainly to do with measurements (metrics) and discipline. PSP manages to make the developers believe in the importance of certain measurements thus reducing the resistance and the overhead of performing Level 4 activities. Furthermore, the measurements introduced by PSP are at the right granularity level so that statistical processing would facilitate the control of the process.
Through the experiment it became evident that a repeatable process infrastructure (CMM Level 2) substantially supported the use of PSP. In our case, relatively simple modifications to existing processes and tools provided the means to smoothly enact PSP.
PSP Training in parallel with the every day workload and commitments has presented major obstacles to the developers which were overcome with the exceptional commitment from their part. However, this can not be considered as the normal case so top management commitment and proper scheduling of the training period are required.
The measurements collected from the use of CL_PSP1 and CL_PSPR1 were in general less than what has been set as target. However, the improvements are still significant and they are certainly promising. Furthermore, the benefits from the PSP are manifold indicating that the systematic use of PSP will improve the performance and maturity of the Omega development process at large.
In our case the use of PSP is justified and recommended in the development of core, critical or highly reusable components, where the cost of errors and the need for accurate time estimations is very large. At the current state, top management is convinced about the ROI benefits of applying PSP on the above type of development.
The formal PSP training is not performed at the initial Omega training but rather is provided as an incentive to best performing developers so that they can be used in critical developments. However, the PSP concepts and the CL_PSPxx processes have been included in the Omega training curriculum.
Future actions include the refinement of CL_PSP1 and CL_PSPR1 and the training of more developers in PSP. Furthermore, we plan to introduce PSP in more processes and activities of the Omega infrastructure such as the Customer Implementation Services process.
Having PSP introduced into core processes of the Omega our following process improvement efforts will be targeted for the CMM Level 4.
Table SME.1: CL_PSP1 Omega Object Categories – Effort Estimation.
Report Element Category
Table SME.2: CL_PSPR1 Omega Report Element Categories – Effort Estimation.
Table SME.3: CL_PSPR1 Defect Types in Report Development Process.
Table SME.4: CL_PSPR1 Defect Types in Omega Development Process.
Chart SME.1: Accuracy of the size estimation throughout the course-works. The two lines should be as close as possible increasing thus the predictability of size.
Chart SME.2: Similar to SME.1 with the difference that it demonstrates the accuracy of time estimation.
Chart SME.3: The percentage for each type of defect injected in the Coding phase. These data would form the basis for the compilation of the appropriate Code Review Checklists. The PSP 1.1 and CL_PSP1 categorisation of errors is shown in Table SME.4
Chart SME.4: The percentage for each type of defect that is removed in the Test phase. These data would guide the construction of the most appropriate Test cases instructions for the testers. The feedback provided to the developer helps him to understand to which type of errors is more vulnerable.
Chart SME.5: The distribution of the defects found in Code Review per defect type. This information along with the data from Chart 4 form the guide lines for the Code Review Checklists.
Chart SME.6: Comparison of defects found in Code Review versus Compile phase. We should try to maximise the number of defects found in Code Review and minimise those found in Compile phase.
Chart SME.7: Percentage of defects found in each phase. The ultimate goal of PSP and therefore CL_PSP1 is to minimise the number of defects found in Test because they are more expensive to fix.
Chart SME.8: The percentage of defects removed in each phase. Similarly, PSP’s goal is to remove defects in the early stages of development.
[1] Watts S. Humphrey, A Discipline for Software Engineering, Addison Wesley. [2] Carnegie Mellon University, Personal Software Process, lecture notes [3] Avratoglou Ch., Process Management Improvement for Software Development, ESSI Project: 21551, Final Report, 25 Aug 1997 (http://www.esi.es/VASIE/Reports/All/21551/Report/index.htm)