Journal:Ten simple rules for developing usable software in computational biology

Full article title	Ten simple rules for developing usable software in computational biology
Journal	PLOS Computational Biology
Author(s)	List, Markus; Ebert, Peter; Albrecht, Felipe
Author affiliation(s)	Max Planck Institute for Informatics, Saarland Informatics Campus
Primary contact	Email: pebert at mpi-inf dot mpg dot de
Editors	Markel, Scott
Year published	2017
Volume and issue	13(1)
Page(s)	e1005265
DOI	10.1371/journal.pcbi.1005265
ISSN	1553-7358
Distribution license	Creative Commons Attribution 4.0 International
Website	http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005265
Download	http://journals.plos.org/ploscompbiol/article/file?id=10.1371/journal.pcbi.1005265&type=printable (PDF)

Introduction

The rise of high-throughput technologies in molecular biology has led to a massive amount of publicly available data. While computational method development has been a cornerstone of biomedical research for decades, the rapid technological progress in the wet laboratory makes it difficult for software development to keep pace. Wet lab scientists rely heavily on computational methods, especially since more research is now performed in silico. However, suitable tools do not always exist, and not everyone has the skills to write complex software. Computational biologists are required to close this gap, but they often lack formal training in software engineering. To alleviate this, several related challenges have been previously addressed in the Ten Simple Rules series, including reproducibility, effectiveness, and open-source development of software.

Here, we want to shed light on issues concerning software usability. Usability is commonly defined as "a measure of interface quality that refers to the effectiveness, efficiency, and satisfaction with which users can perform tasks with a tool." Considering the subjective nature of this topic, a broad consensus may be hard to achieve. Nevertheless, good usability is imperative for achieving wide acceptance of a software tool in the community. In many cases, academic software starts out as a prototype that solves one specific task and is not geared for a larger user group. As soon as the developer realizes that the complexity of the problems solved by the software could make it widely applicable, the software will grow to meet the new demands. At least by this point, if not sooner, usability should become a priority. Unfortunately, efforts in scientific software development are constrained by limited funding, time, and rapid turnover of group members. As a result, scientific software is often poorly documented, non-intuitive, non-robust with regards to input data and parameters, and hard to install. For many use cases, there is a plethora of tools that appear very similar and make it difficult for the user to select the one that best fits their needs. Not surprisingly, a substantial fraction of these tools are probably abandonware; i.e., these are no longer actively developed or supported in spite of their potential value to the scientific community.

To our knowledge, software development as part of scientific research is usually carried out by individuals or small teams with no more than two or three members. Hence, the responsibility of designing, implementing, testing, and documenting the code rests on few shoulders. Additionally, there is pressure to produce publishable results or, at least, to contribute analysis work to ongoing projects. Consequently, academic software is typically released as a prototype. We acknowledge that such a tool cannot adhere to and should not be judged by the standards that we take for granted for production grade software. However, widespread use of a tool is typically in the interest of a researcher. To this end, we propose 10 simple rules that, in our experience, have a considerable impact on improving usability of scientific software.

Rule 1: Identify the missing pieces

Unless you are a pioneer, and few of us are, the problem you are working on is likely addressed by existing tools. As a professional, you are aware of this software but may consider it cumbersome, non-functional, or otherwise unacceptable for your demands. Make sure that your judgment is shared by a substantial fraction of the prospective users before you start developing a new tool. Usable software should offer the features needed and behave as expected by the community. Moreover, a new tool needs to provide substantial novelty over existing solutions. For this purpose, list the requirements on the software and create a comparison table to set the new tool against existing solutions. This allows you to carve out the selling points of your tool in a systematic fashion.

Rule 2: Collect feedback from prospective users

Software can be regarded as providing the interface between wet lab science and data analysis. A lack of communication between both sides will lead to misunderstandings that need to be rectified by substantially changing the code base in a late phase of the project. Avoid this pitfall by exposing potential users to a prototype. Discussions on data formats or on the design of the user interface will reveal unforeseen challenges and help to determine if a tool is sufficiently intuitive. To plan your progress, keep a record of suggested improvements and existing issues.

Rule 3: Be ready for data growth

First estimate the expected data growth in your field and then design your software accordingly. To this end, consider parallelization and make sure your tool can be integrated seamlessly in workflow management systems (e.g., GALAXY and Taverna), pipeline frameworks (e.g., Ruffus and SnakeMake), or a cluster framework (e.g., Hadoop, http://hadoop.apache.org/). Moreover, make sure that the user interface can scale to growing data volumes. For example, consider that the visualizations should still be comprehensible for larger datasets, e.g., by displaying only parts of the data or through aggregation of results.

Rule 4: Use standard data formats for input and output

As an expert in your research domain, you know the established data standards and related programming libraries for reading and writing commonly used data formats. Make sure that your tool’s output follows standard specifications to the letter, but be as lenient as possible when users provide non-standard input. Tools that follow this rule are more likely to become successful. If you are working in an emerging field with no prevalent model for data exchange, provide data in a structured text file (e.g., tab-separated tables, XML/XSD, or JSON) and aim for self-documenting output by including header lines and data type descriptions. In this case, document how users can derive suitable input data for your tool.

Rule 5: Expose only mandatory parameters

Exposing all (possible) parameters to a user can be confusing and carries the risk of nonsensical parameters settings. When possible, users will thus rely on default parameters. The same applies to benchmark studies comparing your tool against the state-of-the-art competitors. This has three important implications: (i) expose only a small set of parameters by default whose effects on results can be easily understood by any user, (ii) offer advanced parameters only in an expert section and describe them thoroughly in the documentation, and (iii) choose conservatively (and if possible, justify) the default values for parameters such that the tool can operate in a wide range of scenarios and within reasonable run time.

Rule 6: Expect users to make mistakes

You should never assume that your tool is self-explanatory, that requirements concerning the input data are obvious, or that the user will immediately grasp all details of the problem at hand. Ideally, your tool supports the user in using it appropriately, e.g., by checking that data remain inside required ranges or that identifiers are unique, and provides descriptive error messages in case of unexpected values. If performance penalties due to such checks are a real concern (which should be tested), make the checks optional and enabled by default. Finally, allow users to stop ongoing operations in case they realize they made a mistake.

Rule 7: Provide logging information

Two types of logs improve usability and also support the user in making their research more reproducible. Configuration logs keep track of basic information, such as the time stamp of the analysis, the version of your tool and of third-party libraries, as well as the parameter settings and input data. Archiving this information is particularly important in long-running research projects in order to trace irregularities in the results at any later point in time. Technical logs, on the other hand, contain progress messages that help users to pinpoint errors in the execution flow and allow clear communication of these issues to the developer. As much as possible, avoid exposing potentially sensitive user information in the logs.

Rule 8: Get users started quickly

Complex setup routines introduce dependency or configuration debt; i.e., the user has to spend substantial time installing software and learning about the execution parameters of a tool. These raise the bar for unhindered exploration of software features. Such issues can be solved by implementing a web application (if feasible with respect to resource demands), by providing a standalone executable, or by providing a system-specific software package. Alternatively, issues of a program’s dependence on third-party libraries can be avoided by encapsulating your tool in a virtual machine image or, e.g., a Docker container (https://docker.com). Finally, it is imperative to provide demo data that enable users to immediately interact with the software. A successful test run proves to the user that your software works as expected and will be essential if you want your tool to be published.

Rule 9: Offer tutorial material

Researchers can seldom afford the time to thoroughly read complex user manuals. They will thus appreciate a number of clearly written code examples, illustrations, or video screen casts to get started. Most importantly, documented use cases enable users to quickly assess if your tool is suited for the problem at hand and allow fast learning by doing. Keep in mind that these materials have to be updated together with your tool.

Rule 10: Consider the future of your tool

For long-term availability of your software, use suitable repositories such as GitHub (https://github.com) or Bitbucket (https://bitbucket.com) throughout the development process. Explicitly state under which software license you release your code for third parties (see https://opensource.org/licenses). Without such a license, using your software might be prohibitive for many organizations or companies. More importantly, keeping your code in a public repository will also allow you to engage with the users through issue tracking (e.g., bugs, suggestions). After releasing your tool, expect support requests and take them seriously. See them as an opportunity to continuously improve the usability of your tool.

Conclusions

In the above ten simple rules, we highlight that software should not only be scientifically sound but also be perceived as usable for widespread and effective application. To these ends, developers should also be the first to apply their tool, to reveal usability issues as early as possible. However, effort is required from both users and developers to further improve a tool. Even engaging with only a few users (Rule 2) is likely to have a large impact on usability, since, as Jakob Nielsen put it, "Zero users give zero insights."

Further reading

Usability is an important topic in software design, and we would like to provide a few starting points for further reading:

• Baxter, S.M.; Day, S.W.; Fetrow, J.S. et al. (2006). "Scientific software development is not an oxymoron". PLOS Computational Biology 2 (9): e87. doi:10.1371/journal.pcbi.0020087. PMC PMC1560404. PMID 16965174. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1560404. 

</ref>

• Borchardt, J.-C. (30 June 2011). "Usability in Free Software". jancborchardt.net. http://jancborchardt.net/usability-in-free-software. 

• Macaulay, C.; Sloan, D.; Jiang, X. et al. (2009). "Usability and user-centered design in scientific software development". IEEE Software 26 (1): 96–102. doi:10.1109/MS.2009.27. 

</ref>

• Nichols, D.; Twidale, M. (2003). "The usability of open source software". First Monday 8 (1). doi:10.5210/fm.v8i1.1018. 

• Seffah, A.; Metzker, E. (2004). "The obstacles and myths of usability and software engineering". Communications of the ACM 47 (12): 71–76. doi:10.1145/1035134.1035136. 

• Sloan, D.; Macaulay, C.; Forbes, P. et al. (2009). "User research in a scientific software development project". Proceedings of the 23rd British HCI Group Annual Conference on People and Computers: Celebrating People and Technology 2009: 423–429. http://dl.acm.org/citation.cfm?id=1671066. 

Acknowledgments

We would like to thank Thomas Lengauer, Nico Pfeifer, and Fabian Müller for their critical reading of the manuscript and insightful comments.

Funding

FA and PE acknowledge the support of the German Federal Ministry of Education and Research grant no. 01KU1216A (DEEP project). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests

The authors have declared that no competing interests exist.

Notes

This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added. In the original conclusion, the authors provided suggested reading, but the original only included them as references. These have been moved and expanded out into a new section after the conclusion called "Further reading" and organized alphabetically by author (but now no longer appear as citations).