Evaluating Open Source Software Systems

According to Dr. Johnson, a high quality open source software system obeys three "prime directives":

1. The system successfully accomplishes a useful task.
2. An external user can successfully install and use the system.
3. An external developer can successfully understand and enhance the system.

To illustrate the difficulty involved in achieving these three criteria, we were assigned the task of choosing a Java-based open source project from SourceForge and evaluating it against the prime directives. It was almost trivial finding projects that were useful and usable. The challenge was in finding a project that provided adequate resources for developers in order to satisfy the third prime directive.

I chose to evaluate the Vuze (formerly Azureus) project because it won Most Collaborative Project in the SourceForge.net 2007 Community Choice Awards. Surely such a collaborative project would make it easy for external developers to contribute to the project?

SourceForge Project Page: http://sourceforge.net/projects/azureus/
Official Project Page: http://azureus.sourceforge.net/
Vuze Homepage: http://www.vuze.com/app
Project Wiki: http://www.azureuswiki.com/index.php/Main_Page
Vuze Forums: http://forum.vuze.com/index.jspa?categoryID=1

Prime Directive 1

Vuze is a peer-to-peer file sharing program that enables users to download and host files using the BitTorrent protocol. In addition to being a BitTorrent client, Vuze also allows users to discover, download, stream, and share high quality video, audio and game content through its Vuze network. You can even publish and monetize your own original content on the Vuze network. Because it provides additional features beyond that of a basic BitTorrent client, Vuze definitely satisfies the first prime directive.

Prime Directive 2

Installing Vuze was very easy, as the project provides pre-packaged installation files for Windows, Mac OS X, and Linux (the uncompiled source files are available as well). From the SourceForge download statistics, the program has been downloaded over 70 million times over the past 12 months, and according to one estimate, it has been installed on over 2% of Windows PCs worldwide. It is safe to say that external users can at least install the program successfully.

I found using the software to be pretty straightforward as well, and I was quickly able to start streaming a video from the Vuze network. The project wiki provides extensive end-user documentation, including an initial setup guide, a user guide, settings and configuration information, tips and tricks, and a FAQ. In addition, the Vuze forums provide a community for questions and further support.

From the installation statistics and the extensive user documentation, it is clear that this project satisfies the second prime directive.

Prime Directive 3

This project lacks developer-level documentation. Although the project wiki has a section for developer documentation, most of it is geared towards plugin developers. The community forums provide some support for developers, but no documentation. In fact, browsing through the forums, I found two threads asking about developer documentation. The answer in both cases was that there is none, and it was suggested that one learn the system by examining the code in an IDE such as Eclipse.

It is worth noting that the project provides good documentation for plugin developers. There is the Plugin API Javadoc; plugin development documentation, including a step-by-step guide and tutorials, on the project wiki; and a plugin developer section on the community forums.

Nevertheless, developing a plugin is not the same as developing for the project itself. Due to its lack of developer documentation, this project fails to satisfy the third prime directive.