Journal:Generalized procedure for screening free software and open-source software applications

Abstract

Free software and open-source software projects have become a popular alternative tool in both scientific research and other fields. However, selecting the optimal application for use in a project can be a major task in itself, as the list of potential applications must first be identified and screened to determine promising candidates before an in-depth analysis of systems can be performed. To simplify this process, we have initiated a project to generate a library of in-depth reviews of free software and open-source software applications. Preliminary to beginning this project, a review of evaluation methods available in the literature was performed. As we found no one method that stood out, we synthesized a general procedure using a variety of available sources for screening a designated class of applications to determine which ones to evaluate in more depth. In this paper, we examine a number of currently published processes to identify their strengths and weaknesses. By selecting from these processes we synthesize a proposed screening procedure to triage available systems and identify those most promising of pursuit. To illustrate the functionality of this technique, this screening procedure is executed against a selected class of applications.

Introduction

There is much confusion regarding free software and open-source software, and many people use these terms interchangeably. However, the connotations associated with the terms are highly significant. So perhaps we should start with an examination of the terms to clarify what we are attempting to screen. While there are many groups and organizations involved with open-source software, two of the main ones are the Free Software Foundation (FSF) and the Open Source Initiative (OSI).

When discussing free software, we are not explicitly discussing software for which no fee is charged; rather, we are referring to "free" in terms of liberty. To quote the Free Software Foundation (FSF):

A program is free software if the program's users have the four essential freedoms:

• The freedom to run the program as you wish, for any purpose (freedom 0).
• The freedom to study how the program works, and change it so it does your computing as you wish (freedom 1). Access to the source code is a precondition for this.
• The freedom to redistribute copies so you can help your neighbor (freedom 2).
• The freedom to distribute copies of your modified versions to others (freedom 3). By doing this you can give the whole community a chance to benefit from your changes. Access to the source code is a precondition for this.

This does not mean that a program is provided at no cost, or gratis, though some of these rights imply that it would be. In the FSF's analysis, any application that does not conform to these freedoms is unethical. While there is also "free software" or "freeware" that is given away at no charge but without the source code, this would not be considered free software under the FSF definition.

The Open Source Initiative (OSI), originally formed to promote free software, refers to it as open-source software (OSS) to make it sound more business friendly. The OSI defines open-source software as any application that meets the following 10 criteria, which they based on the Debian Free Software Guidelines:

• Free redistribution
• Source code included
• Must allow derived works
• Must preserve the integrity of the author's source code
• License must not discriminate against persons or groups
• License must not discriminate against fields of endeavor
• Distribution of licenses
• License must not be specific to a production
• License must not restrict other software
• License must be technology-neutral

Open-source software adherents take what they consider the more pragmatic view of looking more at the license requirements and put significant effort into convincing commercial enterprises of the practical benefits of open source, meaning the free availability of application source code.

In an attempt to placate both groups when discussing the same software application, the term free/open-source software (F/OSS) was developed. Since the term "free" was still tending to confuse some people, the term "libre," which connotes freedom, was added resulting in the term free/libre open-source software (FLOSS). If you perform a detailed analysis on the full specifications, you will find that all free software fits the open-source software definition, while not all open-source software fits the free software definition. However, any open-source software that is not also free software is the exception rather than the rule. As a result, you will find these acronyms used almost interchangeably, but there are subtle differences in meaning, so stay alert. In the final analysis, the software license that accompanies the software is what you legally have to follow.

The reality is that since both groups trace their history back to the same origins, the practical differences between an application being free software or open-source are generally negligible. Keep in mind that the above descriptions are to some degree generalizations, as both organizations are involved in multiple activities. There are many additional groups interested in open source for a wide variety of reasons. However, this diversity is also a strong point, resulting in a vibrant and dynamic community. You should not allow the difference in terminology to be divisive. The fact that all of these terms can be traced back to the same origin should unite us. In practice, many of the organization members will use the terms interchangeably, depending on the point that they are trying to get across. With an excess of 300,000 FLOSS applications currently registered in SourceForge.net and over 10 million repositories on GitHub, there are generally multiple options accessible for any class of application, be it a laboratory information management system (LIMS), an office suite, a data base, or a document management system. Presumably you have gone through the assessment of the various challenges to using an open-source application and have decided to move ahead with selecting an appropriate application. The difficulty now becomes selecting which application to use. While there are multiple indexes of FOSS projects, these are normally just listings of the applications with a brief description provided by the developers, with no indication of the vitality or independent evaluation of the project.

What is missing is a catalog of in-depth reviews of these applications, eliminating the need for each group to go through the process of developing a list of potential applications, screening all available applications, and performing in-depth reviews of the most promising candidates. While true that once an organization has made a tentative selection it will need to perform its own testing to confirm that the selected application meets its specific needs, there is no reason for everyone to go through the tedious process of identifying projects and weeding out the untenable ones.

Illustration 1.: This diagram, originally by Chao-Kuei and updated by several others since, explains the different categories of software. It's available as a scalable vector graphic and as an XFig document, under the terms of any of the GNU GPL v2 or later, the GNU FDL v1.2 or later, or the Creative Commons Attribution-Share Alike v2.0 or later

The primary goal of this document is to describe a general procedure capable of being used to screen any selected class of software applications. The immediate concern is with screening FLOSS applications, though allowances can be made to the process to allow at least rough cross-comparison of both FOSS and commercial applications. To that end, we start with an examination of published survey procedures. We then combine a subset of standard software evaluation procedures with recommendations for evaluating FLOSS applications. Because it is designed to screen such a diverse range of applications, the procedure is by necessity very general. However, as we move through the steps of the procedure, we will describe how to tune the process for the class of software that you are interested in.

You can also ignore any arguments regarding selecting between FLOSS and commercial applications. In this context, "commercial" refers to the marketing approach, not to the quality of the software. Many FLOSS applications have comparable, if not superior, quality to products that are traditionally marketed and licensed. Wheeler discusses this issue in more detail, showing that by many definitions FLOSS is commercial software.

The final objective of this process is to document a procedure that can then be applied to any class of FOSS applications to determine which projects in the class are the most promising to pursue, allowing us to expend our limited resources most effectively. As the information available for evaluating FOSS projects is generally quite different from that available for commercially licensed applications, this evaluation procedure has been optimized to best take advantage of this additional information.

Results

Literature review

Main page: Literature review

Initial evaluation and selection recommendations

Main page: Initial evaluation and selection recommendations

In-depth evaluation

Main page: In-depth evaluation

Completing the evaluation

Main page: Completing the evaluation

Summary

Main page: Summary

Glossary

Main page: Glossary

Notes

This article has not officially been published in a journal. However, this presentation is largely faithful to the original paper. The content has been edited for grammar, punctuation, and spelling. Additional error correction of a few reference URLs and types as well as cleaning up of the glossary also occurred. Redundancies and references to entities that don't offer open-source software were removed from the FLOSS examples in Table 2. DOIs and other identifiers have been added to the references to make them more useful. This article is being made available for the first time under the Creative Commons Attribution-ShareAlike 4.0 International license, the same license used on this wiki.