Describes the software development process in Open Source.
Some issues and how they are resolved.
In all the excitement and fury that surrounds the Open source movement and in no small measure Linux itself, it is often very easy to forget the origins and beginnings of this unique and often misunderstood development mechanism. Today, with Sun releasing StarOffice (renamed as OpenOffice[1]), we are often transported to a time when Netscape bravely ventured[2] where few had dared to go before - to release a commercial application into the open domain. In this three part series we look at the open source model and it's history and the pros and cons that surround this development mechanism.
The history of the term "Open Source" as well as Netscape opening up the source code for their browser is linked in no small measure to Eric S Raymond[3]. He is the author of the authoritative Jargon File[4] as well as the extremely useful fetchmail[5] client, in addition to various other interesting efforts. In his seminal paper titled "The Cathedral and the Bazaar" [6], he describes in some detail how the "Open Source" mechanism helped him create fetchmail, a high quality, fast and effective tool for mail retrieval.
A very long time ago (atleast by Internet time) Fred Brooks wrote a paper titled "No Silver Bullet". Linked to this paper, he wrote a pioneering book on the process of software development titled The Mythical Man Month[7]. In this book Fred Brooks wrote out a simple truth on the problems surrounding software development. Known today more as Brook's law it is often phrased as "Adding more people to a late project only makes it later". In simple direct cause-effect mathematics, this law does not make sense. If there are a 100 people working to write a piece of code, shouldn't adding a 100 more to the project cause the project to finish in half the time? However, the time that is required to integrate the 100 new people into the project, to ramp them up to productivity, to assimilate a 100 new egos into the collective whole will take it's own time. Unfortunately it is a well observed process that adding a 100 people to a project would double the time required for the project if not more. If this is the case in well established software companies with excellent software practices, then is there a solution to this problem?
As Eric S Raymond points out, Linux is a fine example that proves that Brook's law is not complete if certain situations exist. As is common knowledge, Linux was built by volunteers on the Internet, who collectively hacked together a fine production quality operating system with powerful features and a wide and dedicated user base. Linux has continued adding programmers and volunteers with gay abandon, with not a scrap of attention to Brook's law. There is of course an inherent contradiction here - if Brook's law makes so much of sense, then how do Open source projects like Linux, Apache[8] and Mozilla [9] survive and function? The answers are in that as opposed to the conventional rigid and formal style of software development (read as Cathedral building) a free, open and egoless style such as that of Linux (read as a Bazaar) is likely to produce finer results.
Consider this - on the Linux Kernel Mailing List(lkml)[10] Linus Torvalds the originator of Linux recently posted the following:
"I'm a bastard. I have absolutely no clue why people can ever think otherwise. Yet they do. People think I'm a nice guy, and the fact is that I'm a scheming, conniving bastard who doesn't care for any hurt feelings or lost hours of work if it just results in what I consider to be a better system."
in a discussion regarding whether a kernel debugger would result in better code or not. This is not any conventional hierarchy or formal structure. This is an intellectual free for all; a evolutionary computing struggle in which only the best and brightest ideas win; one in which suggestions and help is freely welcome from any quarter and in which rank, order or the position of the proponent make for little value. This is typically the reason why open source projects end up as both technically superior and functionally effective. The end users are part and parcel of the development process, nay they are the driving elements of the process. In such a mechanism, free and uninhibited feedback leads to smarter and better bug fixes at each step. For far too long, open source meant that you had a whole horde of people tirelessly working as beta testers or even worse, as testers for free. To assume that just because more people are looking at the source implies fewer bugs in your code is a simplistic assumption that can often lead to error. Just as an illustration, OpenBSD[11] is often rated as a far more secure operating system than Linux although the user base of Linux far outstrips that of OpenBSD. These and other contradictions we shall look into in greater detail in the next parts.
From the overwhelming evidence that we have at hand, we can deduce that the open source model works. However, we will need to study it in some detail to examine the key factors that drive it. In the next two parts, we shall see exactly how the open source process operates, and some of the problems that are coming to light as we see more use of this method of software development. Till then consider that like a mind, source code can sometimes be more effective only if it is open and not closed.
[1] http://www.openoffice.org/
[2] http://www.netscape.com/newsref/pr/newsrelease558.html
[3] http://www.tuxedo.org/~esr
[4] http://www.tuxedo.org/jargon
[5] http://www.tuxedo.org/~esr/fetchmail/index.html
[6] http://www.tuxedo.org/~esr/writings/cathedral-bazaar
[7] http://www.cs.unc.edu/~brooks/
[8] http://www.apache.org/
[9] http://www.mozilla.org/
[10] http://kt.linuxcare.com/kernel-traffic/
[11] http://www.openbsd.org/