Software Process Improvements in a Very Small Company
Ita Richardson Department of Computer Science and Information Systems and Small Firms Research Unit, University of Limerick, National Technological Park, Castletroy,Limerick,Ireland
Kevin Ryan College of Informatics and Electronics, University of Limerick, National Technological Park, Castletroy, Limerick, Ireland
Authors have recognised that the available models did not provide an improvement strategy [1], [2]. The IDEAL model for the Capability Maturity Model was presented in 1995 [3], but as recently as 1998, Debou and Kuntzmann-Combelles note that "due to lack of documentation on the post-assessment phase, assessments are often being performed as a one-shot event without connection to any improvement strategy" [4]. For the large company, it may be possible to devise such a strategy by investing in the development of action plans. However, for the small company, this requires yet another investment from a much smaller purse.
The authors of this paper have developed a generic method to be used by Small Software Development companies, allowing them to devise Software Process Improvement strategies. This method is based on Quality Function Deployment. They implemented the method – Software Process Matrix - in two small software development companies in Limerick, Ireland. The paper discusses this implementation in one of those companies.
Figure 1
Software Process Matrix
The values given to strong, medium and weak relationships are 9, 3, 1 respectively and on Quality Function Deployment charts they are normally represented by the symbols: l or ¥ (strong), m (medium), and D (weak). To use the Software Process Matrix, companies carry out a self-assessment of their software process and input the results. SPM outputs a prioritised action list showing the practices to implement. A sample of the Software Process Matrix is displayed in Figure 1.
Although Computer Craft started out developing business software itself, in recent years they partnered with a U.S. company, giving Computer Craft the rights to modify and install a software package (Data Organizer) developed by this U.S. company. There are two aspects to the modifications made to this product. Initially, the product was to be localized. For example, Irish and U.K. legislation had to be accounted for. It can also be customized for the needs of the user. If major changes are required by the customer this often becomes a separate module, programmed in Ireland, and retained as a generic feature. This varies from project to project. About 30% of customers require extensive customisation. The Computer Craft engineers install software directly in customer sites.
The fact that both the Group Managing Director and the Software Development Manager were willing to give employee time to the project is significant in itself. However, when recruitment difficulties arose, this project was given lower priority.
Figure 2
Information flow between customer and developer
For customized product, the customer services project manager wrote up a requirements specification. There was no particular standard in use – "each customer services project manager identifies the need in their own way", and complete information was not always received from the client: "the client may not know how to address particular issues". Once completed, this document was signed-off by the customer and was passed to the software development group. However, there were cases where "questions were left unanswered" and "information was lacking".
When the software development group received the requirements specification, a functional specification was written. This specification was reviewed and accepted by the customer. Again, this was done through the customer services project manager. In the opinion of at least one software engineer, they could have "quantified what needed to be done" earlier in the project if someone from the development group met the customer. It was not uncommon for work on the functional specification to be stopped until the customer made decisions about their requirements.
Other difficulties arose with the functional specification, where there "were lots of things missing", "it was different to what was tendered". For example, one problem arose because the person whom the customer services project manager dealt with at the customer site was not the final user of the system. On another project, the software engineer assumed that all the information gathering had been completed for the requirements specification, but this had not been the case.
Customer requirements often changed as developers proceeded through the design stages. One developer stated: "There is a huge amount of information from the customer, but it is changing – moveable goalposts". When changes came from customers, there was no formal means of dealing with them. The difficulty faced by the group was that features not previously included in the requirements and subsequently the functional specification impacted the final project schedule.
During some projects, developers met customers at prototype stage, but this was more the exception than the norm, which was that the developer would meet the customer during installation. Not meeting the customer until late in the project caused problems in the development group. Developers found that "information useful to the design and development was gathered from the customer by the developer during the installation".
As with requirements specifications, there were neither standards nor templates for functional specifications within the company. One developer told the researcher that he wrote a "functional specification based on his own experience". Consequently, each engineer had their own format and layouts and sections within specifications varied considerably. However, when functional specifications were written, the decisiveness of the engineer was usually prevalent throughout them. For example: "The user will have the option to approve, reject or hold a transaction", "after review the user will have a set of transactions that are approved".
Developers sometimes found problems at later stages in development, such as system test and installation, which could have been avoided if requirements had been collected properly. This was a frustrating situation for the software engineers, particularly as they had deadlines to meet.
Following test completion, the software was shipped to the customer services project manager who installed the software for the customer. In some cases the software development manager or software engineer helped with the implementation. When there were changes made during implementation the customer compared the product with the requirements specification, which was unambiguous where they were concerned. Software installation was done either from diskettes or a laptop computer.
In the early stages of projects, the software development manager decided what were the critical tasks from the initial customer requirements, to whom these tasks were assigned and the time-scales involved. Software developers were often working on multiple projects and were seen by the manager as "a team working together". The software development manager was ultimately responsible for the completion of the project.
Group meetings were important within Computer Craft, mainly to keep people informed of progress on development projects. There were two general meetings held each week. One was a production meeting, attended by the two software development group managers (mainframe and PC) and customer services. This was followed by the software development group meeting, at which development tasks were scheduled and updates from the production meeting passed onto the group. Normally, two project meetings were also held each week.
The overall importance of each process was calculated and an example is shown in Tables 1 and 2.
Future Performance
Table 1
Rating Customer Requirements
For the process ‘Establishment of project teams’, current performance of 2.8 is the average score given by the software engineers on the self-assessment questionnaire. The planned future performance of 4.0 means that the software development manger wants to see these being established by applying an organisational procedure. The improvement factor in Table 1 is calculated as: 1 + (Planned Future Performance – Current Performance) * 0.2, giving, in this case, a value of 1.24.
Table 2
Calculating Overall Importance
The importance to the company value of 4.0 means that the software development manager stated that this process was of high importance to the company. As shown in Table 2, the overall importance was calculated by multiplying the importance to the company by the improvement factor. Percentage importance of each process is the overall importance stated as a percentage of the total overall importance in the Software Process Matrix.
According to the results the following were the top nine processes that were important to Computer Craft:
Table 3
Importance of the Practices
The importance of the practice was also stated in percentage terms, and those with the highest values were identified as those that would cause the greatest improvement to the software process within Computer Craft. Examples of practices are in Table 3.
The top ten practices proposed by using the Software Process Matrix as the basis for an action plan were:
Action 1:
Program specifications for this project were written by all members the software development group, all using the available template. The customer project manager examined the documents, and signed off on quotations generated from them. Specifications were passed between developers and all information needed by them was available in the specifications. The document history was updated on some documents the researcher examined, although she was told by the project manager that this was something he did not follow up on and the engineers did this "off their own bat". Failure to update specifications with all modifications from the customer caused some problems during testing.
The existence of guidelines ensured that specifications were consistent, and correspondence indicated that the customer was satisfied with the documentation she was receiving. However, one of the engineers stated that:
"the specifications were detailed – much more detailed than I have ever seen before, in one way this is a good idea, but sometimes you find problems later because the specification has been taken as gospel".
One section of the Banking Organizer project suffered from "feature creep". Because of this, the engineer working on the software found that changes which "should be easy but because of this are relatively difficult".
Software development group meetings were held at the start of each week. At this meeting, the group reviewed the work done and action items closed during the previous week, the target for the current week, action items open, daily work done by individuals within the group for the previous and current weeks, and any off-site visits to be carried out during the current week. This allowed the group to be updated on the status of projects, and gave a formal forum for discussion around problems that existed on any projects.
The second action identified was a combination of setting up procedures to specify and document system requirements and to collect, identify, record and complete new customer requests. This action had been worked on with the development of a specification which included the requirements, functional and technical specification. They also improved their method of dealing with customers, meeting them early on in the development process, updating them on the project and clarifying requests with them. This ultimately effected the test of the product and its implementation. As this had such a positive effect on the processes within the company, it is important that this procedure is maintained for the future, and that the method of dealing with the Banking Organizer project become accepted by the organisation.
Setting up procedures to define delivery contents and to verify all changes to requirements are monitored once the requirements specification was signed off was the third action identified. Two procedures had been written – Installation procedure and Build Release to Customer Services – which covered the first of these. Using these procedures, the implementation of the Banking Organizer went smoothly, with very few problems. While not done formally, changes to the requirements were updated in the specifications when changes were made. This process would need to be better controlled in the future.
Action 4 required that Computer Craft assign a person software quality assurance responsibilities. This had already been done, but their responsibilities were not clear to them. The newly appointed quality assurance engineer, had taken testing as one of her responsibilities, consequently she wrote up detailed test plans and based these on the requirements specification. Her experience was lacking, and it was identified that she needed training to help her fulfill her role within the organisation. Nonetheless, the introduction of detailed test plans had helped the testing of Banking Organizer to be completed efficiently and effectively. A downside to this was that there was little emphasis placed on code reviews within the organisation, and it would be advisable for the company to re-consider this situation. Her other responsibility was the writing and approval of procedures, a number of which were written during the research period. The other practice to be worked on in Action 4 was to define quality criteria and metrics for the project deliverables. This had been partially done, but was not accepted as being an organisation practice.
The fifth action required a procedure to establish product baselines for each product supplied. This was not worked on. It also required that a procedure be set up to assign responsibility for the software development plan, work products and activities. While a procedure was not set up for this, the software development manager took this responsibility on board, and improved the project management within the organisation. This is reflected in the improvements evident in many of the project activities.
Of the ten practices identified, three were not completed during the research period. Another three had been implemented but not formalized with the organisation. It is important that this be done, as the evidence from the Banking Organizer project was that they have aided the improvement of the software process within Computer Craft.
During initial research, there had been no procedures nor guidelines used for the collection and documentation of customer requirements. Documentation and collection methods varied between software engineers, even when they were working on the same project. By the end of the research period, a template for the program specification had been introduced, containing a section on requirements. This replaced the previous requirements, functional and technical specifications. Software engineers working on one project were providing and working from consistent information and the customer was satisfied with the output.
The existence and use of the specifications did not prevent "feature creep", but contributed to its reduction. Unlike the Data Function product, there were no modifications required after implementation of the Banking Organizer. Also, developers could code the system knowing that very few changes would be made.
The implementation of the practices as identified when using the Software Process Matrix was partially responsible for the improvements in Computer Craft’s software process. However, without management support for the project, particularly when the company was going through recruitment difficulties, the identification of the practices alone was not sufficient. It was important that the exercise was followed through and the practices were implemented within the organisation. The software development manager recognized that changes were required and used the Software Process Matrix as a basis for identification of the most relevant changes.
The company has not identified any market requirement for the implementation of any particular standards such as ISO9000 or for being assessed using SPICE or the Capability Maturity Model. It is possible that customers may look for this in the future, and the company is now better placed for proceeding with such actions.
She worked with Wang Laboratories, B.V., Limerick for 8 years until 1991, where she was mainly involved in Programming and Systems Analysis of systems used in the manufacturing plant production, Materials (including MRP), Distribution, Financial, and Personnel. In latter years, she also had responsibility for the maintenance of Systems Standards, and training of Programmers and Analysts.
Her M.Sc. project was a Simulation of the Automatic Storage and Retrieval System used in the manufacturing plant. Her current research involves the investigation of Quality Tools, particularly Quality Function Deployment, and how they can be applied to the software development process.
She was appointed to lecturer in 1998. Her PhD research is in the application of Manufacturing Quality Techniques to Software Process Improvement in Small Software Development Companies. She is a founder-member of the Small Firms Research Unit at the University of Limerick, whose research is specifically concerned with the growth and development of small firms.
Kevin Ryan received a B.A.I. in Engineering in 1971 and a Ph.D. in Computer Science in 1978 all from Trinity College Dublin. His Ph.D. was concerned with the simulation and optimisation of bus scheduling operations. He lectured in Computer Science at TCD in 1972 before working as programmer training manager for RCM Ltd. Zambia where he recruited and trained the first Zambian programmers. In 1979 he was on the faculty of the University of Kansas before rejoining Trinity College Dublin in 1980. He spent 1985/86 as a guest researcher at Linkoping University. In 1990 he became Professor and head of the Department of Computer Science and Information Systems and, since September 1994, has been Dean of the College of Informatics and Electronics.
He has published papers on simulation, programmer training, software methods, expert systems, natural language processing and requirements engineering. His current research interests include systems design methods, knowledge-based software development is currently involved in the EU funded network of excellence in Requirements Engineering (RENOIR).
He has acted as Evaluator for the EU and as external examiner for universities in Ireland and the UK.
He has lectured on Programming Languages, Program Design Methods, Business Computing, Data Structures, Operating Systems, Social impacts of Computing, Software Engineering and Artificial Intelligence.
References
[1] Peterson, Bill, Transitioning the CMM into Practice in Proceedings of SPI 95 - The European Conference on Software Process Improvement, The European Experience in a World Context, 30th Nov-1st Dec, 1995 Barcelona, Spain, pp. 103-123.
[2] Draper, Lee, Kromer, Dana, Moglilensky, Judah, Pandelios, George, Pettengill, Nate, Sigmund, Gary, Quinn, David Use of the Software Engineering Institute Capability Maturity Model in Software Process Appraisals, output from CMM v2 Workshop, February, 1995 Pittsburgh, Pennsylvania, U.S.A.
[3] Peterson, Bill, Software Engineering Institute, Software Process - Improvement and Practice, Pilot Issue, August, pp 68-70, 1995.
[4] Debou, Christophe, Kuntzmann-Combelles, Annie, Linking Software Process Improvement to Business Strategies: Experiences from Industry in Proceedings of SPI 98, The European Conference on Software Process Improvement, 1-4th December, 1998, Monte Carlo.
[1] Clausing, Donald, Total Quality Development, ASME Press, U.S.A., 1994.
[2] Cohen, Lou, Quality Function Deployment, How to Make QFD Work for You, Addison-Wesley, U.S.A., 1995
[3] ReVelle, Jack B., Moran, John W., Cox, Charles A., The QFD Handbook, John Wiley & Sons Inc., U.S.A., 1998.
Further reading on the Software Process Matrix:
[1] Richardson, Ita, Quality Function Deployment - A Software Process Tool? in Proceedings of the Third Annual International QFD Symposium, Linkoping University, 1st-2nd October, 1997, Linkoping, Sweden, pp 39-49, Volume 2.
[2] Richardson, Ita, Using Quality Function Deployment to Develop Action Plans for Software Process Improvement, Proceedings of the 10th Software Engineering Process Group Conference (SEPG ’98), 9th-12th March, 1998, Chicago, U.S.A.