or
tactile aspect alluded to in a section above. It gets more detailed than that
because it seems to be all about who owns the manufacturing knowledge in the
process.Ultimate aim - Software Factory
Background
A software project manager mentioned that it was hard to try and learn up a heavyweight process, let alone implemented it. He commented that RUP 2002 was a large web site (some 38Mb worth). Where should he even start? He implied that there was just too much and with his busy schedule he would simply not get around to it. This meant he was not able to justify implementing it.
Thinking about this, it made sense to compare how a typical large manufacturing company operates, and see if a software development company is different enough not to follow similar principals, in the hopes of discovering a reason to either bother with pursuing the change or not. Intuitively there should be more similarities in process maturity than there are.
Consider a car factory
Modern factories seem to have got the building of a product down to a fine art. Consider for example a factory that builds cars. Think about the process which they use; its iterative, incremental, based upon a time proven, tried and tested architecture, uses project management, configuration management and change management.
Each new car model incrementally improves on the last model. Bugs from the last model are worked upon in research and development and the new fixes are worked into the new design. New requirements are also brought into the new design, be they from external sources like customer research, or internal inspiration from marketing or R&D.
Most new car models are not that far removed from the preceding model. Usually a couple of cosmetic changes, like round headlights instead of square ones, a slightly more rounded aerodynamic body shape as well as something new like a more effective braking system or a safer suspension under extreme swerving conditions. The motor may have had some changes due to international pressure to reduce emissions, and the like.
Occasionally a total rethink is done in order to produce a ground breaking new concept product. But this effort takes a lot longer and probably costs a whole lot more, because of all the extra testing and proof of concepts that have to be undertaken to make sure it reaches market acceptance and conforms to all safety and other standards, which have to be proven from scratch.
Compare the car process to the software process
By comparison software development processes in companies just never seem to reach the apparent efficiency of the processes many manufacturing product factories appear to have.
A class of software company that does appear to be the exception, by being much better at building software than the rest are the software product manufacturers themselves. Companies like Microsoft, Oracle, Borland. Why? Because they have to concentrate on continuously enhancing their base products over and over. They release new products on fixed and relatively regular cycles. These companies obtain requests from their customers to add this and that, fix this and that. The more these companies listen to their customers, the more satisfied the customers feel, which builds better relationships and tends to keep the customers buying the products and not looking for alternatives. Proportionately, though there are not many of this class of software development company.
The vast majority of in-house software developments in large through small corporations come nowhere close to the levels of efficiency and repeatability of process that we have eluded to above, which by the way are in no way perfect. There is always room for improvement in any process.
Why this difference between manufacturing processes?
We all live on the same planet, we can learn across disciplines? There are reasons for the differences, lets explore them and compare them to our car factory. We may discover better practises for the future.
Things change too fast to cope
Many of the reasons probably need minimal explanation. Most of us, even the most recent novice to software development and computing, have seen dramatic change in our lives:
Just when people felt they had mastered a particular environment within which to execute a project, made up of a collection of hardware, development cycle, operating systems, programming language, database, some utilities, certain technologies, development process and using some form of notation, the team manages against all odds to get a project out the door, then the next project comes along and it requires a whole lot of new items from almost every category in the list above.
The result of this never-ending situation is that on every project, everyone is learning and working at the same time, for every project. This creates a much higher risk on the project for 'room to learn and make mistakes', than our car factory, where the next cycle through the process on a different project, not nearly as much has changed. The learning and mistakes from the first time, can be avoided the next few times through the loop because people get better at doing similar activities with experience.
Training in the software development projects therefore becomes a major issue. Managers think they get round this by taking the approach that we hire people who know the technology already. Sometimes they are lucky, some companies projects are ahead of others and people arrive with the experience required.
In reality, many developers spend time trying out the new versions of products, reading up on them while they are supposed to be doing their actual work, so that they know enough to "blag" their way through the next interview on the next project. They then actually get back into the cycle of learning the balance on the subsequent project on the job again.
Occasionally some projects have the budget and the time available to properly train their staff, but these too are few and far between. It is often simply not practical to take a whole team off on a few weeks training to get them all the knowledge they need on all the items mentioned above, and so compromise is the case, and some people only end up getting trained towards the end of the project if at all. Even assuming a developer is lucky enough to have been on a "five day java course" or the like, this may give them the knowledge, but does not give them the experience of having done it to the level required for and in-depth complex project. The incumbent developer still goes through a good few months learning curve to get up to speed in the new technology, fighting through the idiosyncrasies and specifics of the tools, before they become comfortable.
The sort of work we do is not tangible or tactile
While the title above might seem to draw the response "so what?", there is definitely a difference between a tangible product that one spends a days work on, and after all is said and done, steps back away from the object under construction, and there it is in full view for all to see, judge, and react with either appreciation or contempt.
The software developer on the other hand can at best probably only show certain visual differences on the User interface, or a calculation that comes up with a correct answer. There is a sort of incompleteness in much of software development work, which means that nothing can be viewed until the next point of completeness is reached. Either the person must continue working past their normal hours to show this point or must finish early. Typically, neither occurs, and so the management get into a habit of not asking to see more regular progress except on a larger scale, like per release.
Because of the largely invisible product that is being manufactured, different ways of working have emerged, based either on trust or hope, rather than pure metrics. While this in itself is not a bad thing, the unknown is left too hidden and suddenly it comes as a great surprise some weeks later when the project manager expects a fully finished product and the developer shows a small subset of the expected product. Politics, blame, growing mistrust and other types of over reaction tend be the result, if these expectations are not managed in the small and fed out to the different levels of management accordingly.
Remember this scenario happens at different levels; between a developer and his team leader, the team leader and his project manager, the project manager and the customer relations manager, the customer relations manager and the customer and so on up the hierarchy. All because the product is not there in full view all the time, for people to react comparing their internal expectations with what they see the in the product. If the reactions were elicited in smaller more manageable increments, most of the resultant problems would not occur.
Good software development people are scarce and change jobs frequently
Another factor which has exacerbated the problem of the software development process not being as good as a car manufacturing processes, is that of people scarcity. According to the reports one reads, there are more software development projects of all sizes than qualified knowledgeable people who can do them. This causes a high movement of people between companies for numerous reasons; to obtain a better project, offered a better salary, move on because of a project failing, etc.
The good side of this is that many software personnel have worked in many companies, with different processes each time, and have a broader experience generally.
However there are probably more negatives which come out of this equation, depending upon which side of the table you find yourself! Software salaries have skyrocketed. Continuity within companies is lost. Knowledge walks with the people the developers leave. Risk on projects is driven higher, by the very fact that people can leave with the usual working notice period.
Companies become very reluctant to spend money on training software people because of the fact that they may never recover the outlay if the developers leave within a few months of the training. Elaborate schemes are created to avoid this or in many cases, training is avoided altogether - the worst case.
Software People are the machinery. When they leave the process breaks
One of the most obvious differences between the car
manufacturing process and the software development process is the tangible
or
tactile aspect alluded to in a section above. It gets more detailed than that
because it seems to be all about who owns the manufacturing knowledge in the
process.
In the UML class diagram entitled Car Manufacturing on the left, notice there is simple pattern of Car Worker works with Manufacturing Machine which produces Car Product.
In this process the Worker has the Process Knowledge, and the Manufacturing Machine has the Manufacturing Knowledge (coloured in green and shown with a black aggregation relationship).
It is not entirely true that the Machine owns all the manufacturing knowledge, because the Worker would need some sort of training on what to do when and why, but is really only there to oversee and monitor the machines, they know what they have to do. This is also a function of how automated the process is of course.
So in this scenario the Worker operates the Manufacturing machine with his Process Knowledge, the manufacturing Machine manufactures the product with its Manufacturing Knowledge and the Car Product is Produced.
This is done iteratively, per Car Product, with lots of small identical enhancements being done by each machine on a conveyor belt one after the other. Any enhancements made to the Manufacturing Machine program are stored as Manufacturing Knowledge for future newer models.
In this scenario, if the Worker were to leave, another one could be found and trained in only the Process Knowledge. This would not affect the quality of the manufacturing process as this knowledge remained with the Manufacturing Machine.
In the Software Development Process UML Class Diagram below,
notice a very similar pattern to the one in the Car manufacturing process above,
where the Software Worker operates on the PC machine (the tool for
creating the product) which
has as part of it some Development Tools. This combination is used to
compile a Software Product similar to the above with the Car product.
Notice however, the big difference between the Car Worker and the Software Worker in this scenario is the Software Worker owns both the Process Knowledge and the Manufacturing knowledge. (The black aggregation relationship from the green class has moved up to be owned by the Software Worker and not the machine as in the Car manufacturing diagram.)
This Manufacturing knowledge combined with the Process knowledge can be measured in terms of a Skills metric. Please assume in this paper that all the skills are constant for all workers. its not really important how good or bad they are in their job, the fact is where the knowledge of this manufacturing lies.
The conclusion that we can derive from this difference between the Car Manufacturing and the Software Development, is that if the Software Worker leaves the company it has a far more profound impact on the overall software manufacturing process, than the impact of the Car Worker leaving the company. Because once a new Worker has been replaced and shown what to do, the manufacturing knowledge is there already and Cars still get made to the same specifications.
It seems obvious that the more the software industry can move their manufacturing knowledge away from the worker and into the company, the more secure the development process will be from the comings and goings of individuals. By Manufacturing Knowledge read UML Models such as requirement models, analysis models, design models, implementation models, test models, etc.
The software utility tools are definitely moving in this direction, look at Together Control Centre, Compuware OptimalJ and Rational XDE for example.
Taking this to the next level, the more the process knowledge can be built into the company system and machines, and made more readily accessible to workers, the less the workers can take away with them when they leave. This is not to say the knowledge will not move away with the workers, it will because its been memorised in their minds, but it won't only be in their minds, it will still be there for the new workers to see, read and use.
To do.....
Process change is organisational change
To Do....
References
Edwards, Charles. ProcessWave Limited. 2002.
Spence, Ian. Principals of Process Improvement. 2002.