What is a software Process?
Introduction
Its not the products and technologies we are using that are the problem in why so many projects fail. The java compilers work, the databases store and retrieve data, the web servers dish up the pages, the operating systems are ok, etc. Its more about the process than the technologies we are using, which seem to be the problem. We do not seem to understand how to go about organising a team of people to come up with a good, tested, working system. In most cases one person developing their own software can manage quite happily. But as soon as we start adding more people to form a team, it starts getting exponentially more complex and less likely to succeed.
To make it more difficult, the technologies are getting more complex. J2EE, EJB and .NET technology all require learning up while simultaneously getting the developed product out the door. It makes financial sense to leverage the existing legacy applications, which makes the software task still more complex. Modeling using patterns and using UML add yet more complexity again.
How do we navigate all our people through these complex tasks and activities at the right time, for the right reason, efficiently and productively in order to achieve the goal of a working system within budget?
Scott Ambler defines a software process as:
"A software process is a set of project phases, stages, methods, techniques, and practices that people employ to develop and maintain software and its associated artifacts (plans, documents, models, code, test cases, manuals, etc.). Furthermore, not only do you need a software process, you need one that is proven to work in practice, a software process tailored to meet your exact needs."
This definition can be further enhanced by saying:
"A software process is a set of disciplines, phases, iterations, methods, techniques, guidelines, people, tools and practises that a company would employ in order to develop, deploy and support a defined piece of software along with all its associated artifacts (models, plans, documents, code, tests, user manuals, installation instructions, etc.) such that the company's software vision and goal is fulfilled as it changes over time."
Complexity variables
Complexity of a software process is a function of:
Time from start till delivery on a project
The time the team is given from the initial concept being thought up and publicised, though all the financial deliberations and a project go-ahead given. Through until the time that the required software is installed and running having been user acceptance tested and put live and "ready for use". This time is the overall time for the project from the companies and stakeholders perspective, however the actual development team often only get a smaller Window on this time box.
The Time variable is critical to this equation as it is practically fixed, due to market opportunities, competitiveness, etc. It can of course be altered, and often IT management do leave this as the final variable which has to slip if the others are not well controlled.
The Team size is another huge factor affecting the software process. Software can be developed by teams from one through to teams numbering in the thousands. As teams scale, sub-teams within teams form, and the divide and conquer approach groups like disciplines with one another as the team members grow on any particular overall team. Of course the lines of communication exponentiate as the team size grows.
Technologies in and of themselves can become more complex as they evolve, however in many cases while further releases often make the technologies easier to use, there is usually more that it can do on successive releases and this is where the complexity is introduced.
Where the Technology being used is a major factor on the Software Process, is where it is a new technology and the team is still unfamiliar with it, progress is slower and more trial and error occurs. Also a factor is the number of different technologies being used concurrently.
Location of the people forming the team
As the team separates into sub teams in different locations, firstly communication problem factors increase and secondly political factors "us versus them" or culture factors across geographical boundaries make communication harder. This adds to the overall complexity of the software process.
While certain parts of the software process can be split across boundaries, to improve efficiency, this is not always possible. It still remains a complicating factor within the software process.
Rate of change of requirements
As the software development process proceeds over time, business and many other factors continue in parallel. While nothing remains constant, it becomes impossible for those gathering requirements to work with a totally defined set of requirements. Some requirements will fad away, others will be introduced, and all this has an impact on the overall software process.
This particular variable is closely related the to Time variable stated above. The longer the project continues, the greater the likelihood there will be for change and instability of the overall goal. In the worst case, the overall goal of the whole project may alter!
Maturity of the software process already in use
If a company has an existing policy or project in place to enhance the software process, then depending on how far down the line the process has evolved will depend on how mature the process is. Maturity of a company's IT department can be measured at different levels, primarily at the organisational level and also at the project level. This relates directly to the size of the IT team and the overall company.
If for example a company has been though a few projects each time enhancing the process and forcing a learning feedback loop, then the maturity of the process will probably be a lot more advanced and on a higher level of maturity than a company who is trying to adopt a formal software process for the first time on the current project.
Often the size of the budget has an indirect link to the complexity of the software process. The logic follows something like this; We are a huge company and we need this software developed in n months, we don't care how this is achieved, we have thrown enough money at the problem, just get enough good people and make it happen!
What happens here is that the budget indirectly forces the speed of the process to accelerate beyond the time that it would have normally taken. Or it forces the number of members in the team to be increased above that which the project manager would have liked.
On the flip side of the coin this can have a positive effect on the software process because it could mean there will be additional resources put into enhancing the software process. Assuming this relationship is realised by management.
This covers a multitude of issues, but essentially boils down to the type of individuals that make up the team, how they relate as a unit and the work ethic of the team which comes out of this combined group of people. Some people really want to make projects happen, even if the expectations seem unrealistic, they take it as a challenge. Others are inherently negative and will always find ten reasons why this cannot be done. Work ethics vary by country and city. In some areas people are really glad to have work and will take on all sorts of pressure just to get a salary, while in other areas people get work easily and are well paid, and feel they can leave if the going gets tough.
This has a direct impact on the success of any project and in so doing the software process itself. Motivation has a relationship with the team member culture as above. Well motivated teams will work around process problems or invent new ways of succeeding, while the poorly motivated teams will use the process as one of the excuses for not succeeding.
Tools used to support the process
The software process is dependant upon all sorts of software tools being used from simple editors and word processors, through compilers, testing software, change management software, configuration management software, backup and archive software, project management software, web servers, databases, email, etc. If the correct tools are not supplied or the interoperability between various products are not easy, then this has a direct impact on the software process complexity. This includes sound hardware and networking configurations too.
References
Ambler, S. W. 1999-2000. Enterprise Unified Process: Enhancing the Unified Process to Meet the Real-World Needs of Your Organization.
Edwards, Charles. 2002. ProcessWave Limited.