Simple RUP Implementation Evaluation

Background

This Spreadsheet was initially used to try and get a handle on which of the six best practises we needed to look at improving first as we started our evaluation of the team.

Later it was enhanced to compare two companies level of "RUPness" and to try and see areas of strength and areas of weakness, in an effort to get an iterative process improvement technique going. We used one Worksheet Tab for each company and then compared them on the First Tab.

Basis of the Metrics

While the metrics can arguably be said to be relatively subjective, because one person typically evaluates the team and puts down their feel for the answer to each question, the basis of the six best principals are sound, and the individual questions are each able to be mapped onto either a Role, Activity, Tool or Artifact.

Metrics

While at a Rational 2000 Conference a chap called Ed Humphrey talked to a paper called "Model Driven Component Testing" (VM16). In it he showed a Class model of a simple system, and rated each Component/Class a percentage rating as to its individual reliability. If each item is say above 90% reliable on average, one would intuitively expect the overall to be averaged out at somewhere in the same region of 90%. Something like this:

He then went on to say that in combining all these individual components, one could work out the reliability of the system as a whole, multiplying their individual reliability metrics together to give an overall combination of reliability of the system. As you can see the total of each component multiplied out works out to:

.80 x .90 x .95 x 1.0 x .9 x .9 x .95 x .75 x .8 = .32!! A stunning 32%, which is way below what one would intuitively expect.

We used the same logic in this spreadsheet as we contest that each best practise is a "system" that interacts with the other systems to achieve overall functionality.

Areas of enhancement

• This spreadsheet could be adapted to be filled in by many people who know the team, to try and get it less subjective.
• Possibly it could be made more scientific if there ware a balanced number of questions per section and a fixed number per Role, per collection of Artifacts, per set of Activities, tools, etc.
• Possibly assign a Boolean answer to each question and make the question a more simple Yes/No type.

Conclusion

This was really done very quickly in the absence of anything else. Please send in comments and let's enhance this simple model to be more accurate and useable across many projects.

References

Humphrey, Ed. Model Driven Component Testing. Rational 2000.

Edwards, Charles. ProcessWave. March 2002.