Finding a Practical Approach to Organised Reuse
Roar Tørlen PROVIDA ASA, Ålesund Ulf J Krystad PROVIDA ASA, Oslo
For a company like Provida, being a small/ medium sized enterprise operating in a high-competitive international market, it is important to offer flexible and modularised systems within short notice in order to stay competitive. We believe that our process improvement steps focusing on reuse will contribute substantially to this.
Through our participation in a PIE (Process Improvement Experiment) project we have experienced the need for addressing certain key aspects; roles and procedures regarding reuse in practice to be defined and properly implemented into the organisation; the needs for education and training to be focused when making such a paradigm shift; the definition of the requirements for a repository from a reuse point of view to take place.
When introducing organised reuse one should take special precautions to avoid some typical traps. Some of them being underestimation of the need for consensus in the organisation for the change, not having a plan for implementing the new roles and procedures into the standard development environment, and underestimation of the need for some constancy in the surrounding environment while performing the change.
Even if this is a long-term investment and the payoff is expected to materialise in coming projects, we already now see clear evidence that organised reuse gives a shorter time to market.
Provida Retail Division develops large systems for retail and corporate banking, including Customer Information, Loan, Deposits, Corporate Accounts, and Card Management Systems, marketed in Europe, South America and Australia, under the umbrella ProRetail.
ProRetail is marketed as an international basic version, a version for each country and a customer-specific version. A new delivery of the system was always based on a copy of the best-suited existing version. Changes to this version were done according to national or customer-specific requirements, creating a complete new version of the system, which was maintained separately. We realised that this practice would increase maintenance costs dramatically on a long-term basis. Furthermore, we experienced that the development costs for each delivery were too high and that we were not able to take full advantage of previous development in a new delivery. This problem arose since the division was not properly organised to address the aspects of reuse.
One way of improving our development process was to introduce and test out organised reuse, using the method set forth by the ESPRIT project REBOOT (project no: 7808). With financial aid from the EUROPEAN COMMISSION, DGIII-F3, the ESSI program, experiments were carried out on development projects: organised reuse were introduced as the technology being used, part of extra costs incurred by the switch in technology being supported and financed by the Commission. The PIE project was named REPRO (REuse PRocess and Organisation improvement experiment).
Following REBOOT, it was planned that REPRO would work directly with two baseline projects, focusing on development for reuse and development with reuse respectively. We wanted to show that the introduction of new roles, procedures and tools in the first baseline project would contribute to creation of libraries of reusable components that could be reused in the second baseline project. Furthermore, we wanted to show that this strategy was cost-efficient by using metrics and measurements considering all aspects of the development process. REPRO’s role in the baseline projects would be to direct and assist in the adaptation of our new procedures, as well as performing quality control on the results.
The Experiment should introduce organised reuse by focusing on the following key reuse areas:
To be more specific; Requirements were specified, managed, and maintained in Lotus Notes databases. Projects were defined using standard templates in Process Engineer, and scheduled in Microsoft Project. All activities were tracked, and earned value calculated and reported for all deliverables. Quality control was performed in the project for each deliverable and external formal review was performed for all major phases. All units were under configuration management control from unit test and onwards. All future changes were handled according to formal procedures.
For development we used Microfocus COBOL Workbench for programming, Datamanager from MSP as the repository and CCC from Softool as the change and configuration control system. Datamanager and CCC were not integrated, and both tools run on a mainframe with a character-based interface. We clearly saw that the development process suffered under this.
We had grouped our development department according to business domains of our products. However, the personnel were owned by a single project, where the project tended to focus on local goals instead of corporate product strategies. Personnel, solely responsible for reuse, did not take part in the projects. Hence, reuse-specific activities were not incorporated in project plans and reuse practices were at an ad-hoc level, dependent on the individuals in the projects.
Figure KryTor.1: Improvement interaction with baseline project
The yardsticks for how we wanted to measure any results are listed below. We wanted to measure an overall process improvement on the basis of a reuse assessment report in the preparation phase and in the evaluation phase of the project. The assessments were to be based on RMM and should describe our reuse practices before and after the PIE (Process Improvement Experiment).
The definition of new procedures and the use of specific roles, should be a result from the work done with regard to reuse strategy and role and procedure description.
This could be measured by the number of components built. The actual reusability was to be measured in the "development-with-reuse" phase to see how often a component is reused and what changes are needed to use a component in another context. The time-effort spent to tailor existing components to meet new requirements, together with the time spent to build reusable components and the effort saved in reusing them, should be the major parameters in the cost-benefit analysis measuring the value of the libraries.
Originally, we wanted to choose a part of the project as a basis for experiment – called PIE domain. It was more realistic (and practical) to use the new methodology for a complete project. Hence, the PIE domain was the whole project
Retail with Card and Loan. ProRetail is a banking system running on an MVS mainframe. The strategic target sub-systems are CICS and DB2. At this point two products, ProCis and ProDeposits had been developed to that platform. The other products in the ProRetail family, ProCard and ProLoan, were running on IMS/ DL1 platforms. The purpose of this project was to
Figure KryTor. 2 Modularization using domains
The baseline project adding Card and Loan to this architecture, had a budget of 3 mill. ECU and involved 25 people. The project was started late May 96 and lasted for about 12 months. Our main focus was on the design phase and a small part of the Cut (Construction and Unit Testing)-phase.
Figure KryTor.3 Project Gantt Chart
In Figure KryTor.3 solid bars are baseline planned work-packages and shaded bars are actual.
Early impact on organisation.
Our main achievement in this phase was the focus REPRO had got in the organisation, with attention from managing director and down to programmers in the product department. The emphasis we had in REPRO on organisational issues already made an impact on the organisation as such; REPRO defined new roles that were implemented in our product department - not only in our baseline project. The programming personnel in the product department became organised around the domains. This assures high level of competence through all phases of a project. Furthermore, the different groups were capable of supporting several projects at the same time, incorporating requirements from different customers.
Repository decision is strategic.
Furthermore, we became aware of the fact that it was not possible to change repository for one single project (i.e. the baseline project of REPRO) leaving the production line behind using the old tools. This is due to the fact that all information in our current repository is inter-related and that introduction of a new tool for a certain area would lead to major migration problems later. Hence, the first conclusion was that to change the tool for repository was a huge job, well beyond the scope of REPRO.
The Repository was planned to contain documentation, metrics, and configuration management information in an integrated manner. At this stage it was decided, for the time being, to continue with the same repository tools as we had used the last years.
For documentation we continued to use DataManager. For Configuration Management we continued to use CCC. To gather metrics – for which we have had no tools earlier - we used a new project repository, PG4.
Two baseline projects merged into one.
To a certain extent we changed the objectives of the second phase. The reuse assessment had clearly showed that a natural way for Provida to achieve reuse is not an approach where some projects develop for reuseand other develop with reuse as outlined in the original plan.
A deeper analysis than the one we could perform the year before showed that Provida’s ideal organisation for reuse development was somewhat different than assumed. We typically have Integrated development for reuse and are reusing the components in similar projects within the same domain (see section 2.2 of [Kar]). This is also reflected in the organisation which is typically domain-oriented (see section 1.4.4 of [Kar]), where a Product department co-ordinates the work of all delivery projects.
Hence, we focused around a domain in the baseline project. This analysis secured that the development phases of the baseline project were according to the requirements set by REPRO. Thus we guided the baseline project with respect to:
The major adaptation was:
At this point savings of 30 % were reported achieved on the logical level. Our reuse strategy was adjusted to address this saving potential focusing stronger on the product owner role. Since our preparations based on REBOOT focused on reuse at the physical level, we had a problem to define proper metrics on the logical level.
Cancelling the migration to Object Oriented COBOL.
It was clear that neither our customers needed it nor was our development department ready for this step. It seemed to be to many unknown parameters for us to cope with at the same time.
New repository for documentation in place at end of the experiment.
Based on our earlier requirements, a new repository tool, Rochade, was bought for the documentation purpose of the repository. It did not, of course, fulfil all our requirements, but was mainly selected due to:
The tool was used in our baseline projects in Repro. However, we could not map this description to our system model, which resulted in difficulties later when the documentation should be converted to our repository. Thus it was decided not to go along with this way of working.
Project Gateway is a system for building and maintaining project repositories using Lotus Notes.
Our intention was to use this tool to gather all the necessary metrics, and also use it as a project information tool. This turned out to be a too ambitious goal.
However, we actively use the tool to publish project schedules made with MS-project. We also use it for the individual project member to report hours used and outstanding on individual tasks. This way we have an almost automatic project schedule tracking, and it has given us detailed measurement of effort used on detailed activities, sometimes down to physical module level.
As a part of evaluation of repositories we bought an already existing report [Ovum]
For details with regards to Rochade we refer to this evaluation.
The Reuse potential was calculated using the knowledge on the logical functional level, before a mapping was made available on the physical module level. (In fact, that mapping will not be available until the detailed physical design in the Development with Reuse phase).
On the logical functional level under each domain, all functions that need changes or amendments, and all new functions, were listed together with an estimate of the effort of the change. It was further mapped if the functions were needed in the Card, and/or in the Loan development.
From this mapping, the development cost for both Loan and Card separately and together, was calculated.
The findings from this analysis is shown in Figure KryTor.4:
Figure KryTor.4 : Cost/benefit findings
The table shows that for the cut stage, a net saving of 488 days are saved in developing these systems on the common retail platform, rather than as two separate projects on that platform. This is a net save of 19% if we regard the variability analysis as only needed for this purpose. The fact is, however, that much of the work in the V&G must have been done in each project - totally up to 70%. If we take that into consideration, then 237 days of the V&G was needed anyhow (to describe requirements), and the real save is 725 days, which is 28% of the total.
The real save will be higher because of considerable effects in system test and later on in the maintenance phase.
Based on the above analysis, it was recommended (and decided) to develop those systems in common, wherever appropriate.
That is:
After we have developed both the Card and Loan systems, we have run some measures on our resulting (new) repository.
The result is shown in Figure KryTor.5.
Figure KryTor.5: Reuse Numbers
As shown in Figure KryTor.5, the ProRetail product family consists of ProCis, ProDeposit, ProCard and ProLoan. All of which are now converted to our new environment and technical architecture.
ProCis is used in ProDeposit, ProCard , and ProLoan as well as a stand alone system. ProCis contain 1.081.633 lines of code, and the other modules all together contain 2.638.158 lines of code.
We first developed ProDeposit, then ProCard and ProLoan. In Figure KryTor.6 the percentages of new code are shown.
Figure KryTor.6: Reuse Numbers
Since the measurement was taken after all systems were developed, we have, however, not counted for the amendment of code in each system. When we developed Card, we amended some domains in Deposit, and when we developed Loan, we amended some code in both Deposit and Card (because of our tactic – as stated above – the amendment to Card was minimal). The real reuse would be lower than the table indicates. However, even if the amendment is as high as 30%, we have a reuse ratio of more than 50% in both the Card and the Loan development.
That is almost twice as high as we expected!
If required, internal cost models will be changed to stimulate reuse.
We also had a goal to establish a reuse board which should on a regular basis review and guide the reuse activity in our company. This reuse board has not been established. Our experience is that reuse must be an ongoing activity in the professional staff, and hence this is one of the obligations assigned to the group leaders for our product domains.
Also the long-term funding of the reuse activity must be within each project, giving payoff in each project.
Reuse is reflected in all phases of the development process, from requirement specification, through analysis, design and construction to testing. Hence we have:
Maintenance done in one configuration was not built into other configurations because we had no direct reference where we could find out which amendments were needed to those configurations.
The REPRO experiment has changed this behaviour in our division. "Reuse" is now a common known aspect, and is focused on in every discussion regarding productivity. Project planning and start-up activities, focus on how to reuse methods, procedures and components to get the job done in the best possible manner. This reuse thinking emphasises the reuse both on the traditional individual level and on the formal level.
Maintenance is always built into the last common base of the product, and where found necessary – offered to other configurations as well.
Companies having most of the characteristics described below, could probably use our results in a Process Improvement Project introducing reuse:
To start a reuse experiment on the physical level – like outlined in reference REBOOT – we need an organisation with strict conformance to detailed procedures. It is likely to expect this only in ISO – 9000 certified organisations, or in organisations having a similar quality system. The reason for this is that the change in roles and procedures is a huge step. It is not likely to succeed if all these changes are imposed as one step – the organisation needs time to get used to thinking "reuse". Also gathering all the required metric for each physical module is a huge change, if this is not already part of the development process. The motivation will not be present until the organisation understands and think "reuse"
If this is needed for the organisation, it should be anticipated as a major task. It will take time to plan the introduction, to train the staff, and to do the actual conversion. However, an appropriate repository tool is necessary for reuse, either this is on the logical or the physical level
By referencing to organised reuse and a model for maturity we achieve interest from the market.
Once properly learned, we see clear evidence of organised reuse giving significantly shorter time to market. Prototypes are more easily built and thus new user requirements met.
New Roles. Be prepared to focus on new roles in your organisation. Roles such as architect, mentor and reuse expert are the most obvious.
Reuse will not happen all by itself. It is essential to incorporate reuse in the standard development process, and thus avoid dropping out after your first project.
Focus on reuse as a long term investment and not as a short term benefit. However, to get top management support, it is necessary with payoff in following projects. Try to achieve integrated reuse.
Secure skilled people. The bottom line is the availability of good skilled people, and they are hard to find. The Norwegian educational system does not typically produce candidates that can be set to work directly. Additional internal education and mentoring is necessary for increasing the skills in a way that can produce a component based-system. We feel that REUSE should be focused in the education at the university.
At the start up of Development for Reuse we got some trouble in starting up the baseline project, as the contract for the baseline project was not signed as planned in REPRO. This delay also had financial effects of the company, and management attention was shifted more towards the daily operations.
These two events, solving the strategic reuse problem and a difficult market situation, was felt like shifting the focus from the Reuse project. In later phases the project participants felt they no longer had this strong management commitment.
Also the effort of the people involved in REPRO was not longer recognised by the company management, and thus needed support vanished.
We have to take the rest step-wise in a time frame the organisation will accept. Introducing formal measurements of processes after projects and applying metrics to measure the improvement should be the next steps forward.
In REPRO we introduced strict management tools and methodology to the baseline project. This has resulted in very good tracking of projects in the product department.
Some Traps to Avoid
A certain amount of power through a suitable position in the line hierarchy is also important.
In Figure KryTor.7 our current development environment is shown. PG4, Our test script database and the CR/PTD database all are implemented in Lotus Notes. The experiment have added the PG4 database into this environment to capture effort measures, and the main repository tool is changed.
Figure KryTor.7: Tools in our development environment
Organisational Amendments
During this experiment we have improved our organisation with respect to reuse, and introduced reuse roles and responsibilities in the organisation. The target to get to level 3 in RMM is not met on all five key reuse areas. We are on level 3 for both organisation and project management. Also our development process has improved considerably and is between 2 and 3, but in the areas for repository management and metrics we are still about level 2.
Figure KryTor.8: Main areas of reuse
The main areas of reuse are shown in Figure KryTor.8.
Our way of practising reuse is based upon integrated development for Reuse, as described in reference [Kar]. We are currently focusing on the Data Model and our Logical Description, but have also started the process for physical modules. In more details:
For metrics on modules we have to find suitable tools to do our metrics, before this is incorporated into our ordinary working practice.
For future amendments we will use a more involving process from people executing the established roles. In this way we expect a more rapid adoption of a new working practice. Also such amendments should be a result of the work done with regards to reuse strategy and role and procedure description.
Currently an evaluation is going on in order to search for the possibility of moving or version control system from our main frame repository to one on PC. One scenario is to enhance our repository, Rochade, with such functionality.
Build libraries of reusable components was one of our goals in this experiment. This has not been achieved for modules. Currently these modules have a general name, and not a name placing them in one specific domain according to our naming convention.
We will strengthen our architectural role and assign reuse responsibilities to this role. As a starting point we will use generality analyses on components as integrated part of our development procedure. Applying metrics on this level will be introduced on a later stage.
Our development environment is illustrated in Figure KryTor.7. Introducing changes as described above, will bring us a step further to our target solution, illustrated in Figure KryTor.9. However it will still be some future steps to take until we have a totally integrated solution. Figure KryTor.9 is almost the same as anticipated in ref. [Flaa], so we believe there are many case tool suppliers working to reach this solution.
Figure KryTor.9: Target solution for our environment
This has both increased our overall productivity and thus reduced time-to-market for our product. As the market products wary in functional scope, this is not easy to estimate. A reduction on about 30% should be a fair guess. We will try to enhance our cost-pricing model and thus "prove" the validity of organised reuse in business.
The improvement in quality will first show over time, as we get experience with maintenance. Currently we do not have any estimate on this area, but we will continue collecting data.
We have decided to take the division into separate domains a step further, making some of the domains especially in our batch part of the product smaller. This will allow for better tailoring the system according to customer need.
CMM
Capability maturity model - A general process assessment model developed by Software Engineering Institute. In this paper we use it with extentions defined in chapter 5.4.3 of [Kar]
CM
Change Request – describes a separate change to the system
FCM
Factor-Criteria-Metric – A model for software assessment.
This model is described in chapter 4.3.3 of [Kar]
JSP
Jackson Structured Programming
PTD
Problem Tracking Document – describes a problem (or an error), its resolution and to which configurations the problem is fixed.
RMM
Reuse maturity model – A model to assess the reuse maturity of a company.
This model is described in chapter 5.4.2 of [Kar]
V&G
Variability & Generality Analysis
[Kar] Karlsson, Software Reuse – A Holistic Approach, Wiley, 1995
ISBN 0 471 95489 6
[Ovum] Repositories and Framework, Rosemary Rock-Evans
[Flaa] Foundations of Business Systems, Andersen Consulting Arthur Andersen & Co.
Mr. Roar Tørlen is a chief consultant at PROVIDA ASA.
He is born in 1947 and graduated from the Technical University of Norway in 1970, Department for Information processing, and has thirty years of experience in developing information systems.
In the years 1970 to 1975 he worked at the Computing Centre research institute at the university with system development methods and data base systems. In 1975 to 1979 he was with Kongsberg Våpenfabrikk, responsible for systems running on minis handling manufacturing logistic. In 1980 he joined Sunnmørsbanken (later merged with Cristiania Bank og Creditkasse) and has since than worked with banking systems.
He has broad experience in many areas as System Analysis, System Design, Data Base Design, Operation Automation, Technical Architecture, Project Management and General Management. He has been a project manager responsible for several system development projects up to hundred man-month of effort.
In 1995 he joint Provida as a project manager in the Retail division.
Mr. Ulf J. Krystad is a chief consultant at PROVIDA ASA.
He is born in 1953 and graduated (M. Sc.) from the Institute of Informatics, University of Oslo in 1982.
From 1982 until 1986 he worked within a Norwegian CAD/CAM company developing systems for the offshore market. From 1986 until 1989 he worked at IDA building financial systems. In 1989 he joined the department of Industrial Mathematics at SI (Centre for Industrial Research, joined with SINTEF in 1995) as a researcher. In 1999 he joined Provida with main responsibility for the development of clients within the Retail division.
His experience is covering different Management responsibilities, a variety of customer related activities including requirements and specification, lectures and presentations especially within mathematical and numerical modelling, system development and process improvement experiments.
As a leading International Software and Consultancy house, Provida have served the needs of the Financial and Banking Industry for over 30 years. Through creative systems, harnessing modern technology, we have established ourselves as a market leader for these products.
Provida has a wealth of technical and business knowledge, gained through long associations with major financial institutions. With in-depth knowledge throughout the organisation and with the strength of our systems, Provida strives to give our customers the competitive edge needed to succeed in the banking and financial industry.
Provida- Business Areas
Provida develops and sells software solutions and consultancy services to the International Banking and Financial Market. The vision of the company is that Provida will, via its system solutions, contribute to the overall profitability and efficiency of the bank and financial institution.
The main business areas and therefore product offerings are best summarised by referral to the following diagram:
Provida ASA – Business Areas
Retail & Corporate Banking Product Suite
ProRetail represents a new generation of banking systems, and has been developed with a market-oriented banking philosophy and modern system architecture. Emphasis has been placed on the design to provide a structure which, as well as addressing today's business requirements, allows a cost-effective integration of essential new business functions and technologies in the future.
The solution includes up-to-date banking applications for the personal and corporate market. The core systems in ProRetail consist of a central customer, agreement and product administration system plus systems for administering loans, deposits and cards.
Priority is given to real-time access to all customer information. This enables the financial institution to have continually updated information concerning agreements between the customer and the financial institution, the customer’s financial status and all other aspects of a customer’s relationship with the financial institution.
By using the Product Warehouse, the financial institution can create and develop new banking products on-line. This provides a unique opportunity to promote the right products to the market at the right time. This solution enables the financial institution to launch aggressive sales strategies, identify groups of customers with specific needs, and carry out banking operations at a far lower cost than was previously possible.