Journal:Design of generalized search interfaces for health informatics

Abstract

In this paper, we investigate ontology-supported interfaces for health informatics search tasks involving large document sets. We begin by providing background on health informatics, machine learning, and ontologies. We review leading research on health informatics search tasks to help formulate high-level design criteria. We then use these criteria to examine traditional design strategies for search interfaces. To demonstrate the utility of the criteria, we apply them to the design of the ONTology-supported Search Interface (ONTSI), a demonstrative, prototype system. ONTSI allows users to plug-and-play document sets and expert-defined domain ontologies through a generalized search interface. ONTSI’s goal is to help align users’ common vocabulary with the domain-specific vocabulary of the plug-and-play document set. We describe the functioning and utility of ONTSI in health informatics search tasks through a workflow and a scenario. We conclude with a summary of ongoing evaluations, limitations, and future research.

Keywords: information search, search tasks, health informatics, interface design, ontologies, machine learning, PubMed

Introduction

Health informatics is concerned with emergent technological systems that improve the quality and availability of care, promote the sharing of knowledge, and support the performance of proactive health and wellness tasks by motivated individuals. Subareas of health informatics may include medical informatics, nursing informatics, consumer informatics, cancer informatics, and pharmacy informatics, to name a few. Simply put, health informatics is concerned with harnessing technology for finding new ways to help stakeholders work with health information to be able to perform health-related tasks more effectively.

Users in the health domain are increasingly taking advantage of computer-based resources in their tasks. For instance, a 2017 Canadian survey found that 32% of respondents within their last month had used at least one mobile application for health-related tasks. Even more, those under the age of 35 are twice as likely to do so. Furthermore, studies have calculated that over 58% of Americans have used tools like Google and other domain-specific tools to support their health informatics search tasks, with search being one of the most important and central tasks in most health informatics activities.

Yet, search can be challenging, particularly for health informatics tasks that utilize large and complex document sets. For such tasks, health informatics tools may require the use of domain-specific vocabulary. Aligning with this vocabulary can be a significant challenge within health tasks, as they can involve a lexicon of intricate nomenclature, deeply layered relations, and lengthy descriptions that are misaligned with common vocabulary. For instance, one highly cited medical research paper defines the term “chromosomal instability” as “an elevated rate of chromosome mis-segregation and breakage, results in diverse chromosomal aberrations in tumor cell populations.” In this example, those unfamiliar with the defined term could find parsing its definition just as significant a challenge as the term itself. Thus, when communicating across vocabularies, users may struggle to describe the requirements of their search task in a way that is understandable by health informatics tools. To deal with this challenge, ontologies can be a valuable mediating resource in the design of user-facing interfaces of health informatics tools. That is, ontologies can bridge the vocabularies of users with the vocabulary of their task and its tools. Yet, the use of ontologies in user-facing interface design is not well established. Furthermore, health informatics tools that present a generalized interface, one that can support search tasks across any number of domain vocabularies and document sets, can allow users to transfer their experience between tasks, presenting users with information-centric perspectives during their performances rather than technology-centered perspectives. For this, there is a need to distill criteria that can guide designers during the creation of ontology-supported interfaces for health informatics search tasks involving large document sets.

The goal of this paper is to investigate the following research questions:

• What are the criteria for the structure and design of generalized ontology-supported interfaces for health informatics search tasks involving large document sets?
• If such criteria can be distilled, can they then be used to help create such interfaces?

In this paper, we examine health informatics, machine learning, and ontologies. We then review leading research on health informatics search tasks. From this analysis, we formulate criteria for the design of ontology-supported interfaces for health informatics search tasks involving large document sets. We then use these criteria to contrast the traditional design strategies for search interfaces. To demonstrate the utility of the criteria in design, we will use them to structure the design of a tool, ONTSI (ONTology-supported Search Interface). ONTSI allows users to plug-and-play their document sets and expert-defined ontology files to perform health informatics search tasks. We describe ONTSI through a functional workflow and an illustrative usage scenario. We conclude with a summary of ongoing evaluation efforts, future research, and our limitations.

Background

In this section, we describe the concepts and terminology used when discussing ontology-supported interfaces for health informatics search tasks involving large document sets. We begin with background on health informatics. Next, we examine machine learning. We conclude with coverage of ontologies and their utility as a mediating resource for both human- and computer-facing use.

Health informatics

Health informatics is broadly concerned with emergent technological systems for improving the quality and availability of care, promoting the sharing of knowledge, and supporting the performance of proactive health and wellness practices by motivated individuals. Initially, the need for expanded health and wellness services stemmed from rising population levels combined with the growing complexity of medical sciences. These issues made it challenging to maintain quality care within increasingly stressed medical systems. Thus, a central objective for health informatics is the development of strategies to tackle large-scale problems that harm trained medical professionals’ ability to perform their tasks in a timely and effective manner. For instance, telehealth services allowed doctors to practice remote medicine, providing care to those without local medical services. Another early innovation was standardized electronic health care records (EHRs), where patient records were given standardized encodings to provide an increased ability to track, compare, manage, and share personal health information. Some examples of current research directions are the push for stronger patient privacy, personalized medicine, and the expansion of healthcare into underserved regions and communities.

The rising production and availability of health-related data has resulted in a growing number of data-intensive tasks within health. Both private and public entities like health industry companies, government bodies, and everyday citizens are turning to health informatics tools as they manage and activate their health data. A growing number of health-related tasks involve searching document sets. During these tasks, the aim of the user is to use the information described within their document set to increase their understanding of a topic or concept. For example, a search task could be a practitioner searching the EHRs of their patients, a member of the general public using public materials for their general health concerns, or a researcher performing a literature review. In general, a search task involves the generation of a query based on an information-seeking objective. The computation systems of these tools then use this query to map and extract relevant documents out from the document set. Powerful technologies like machine learning are increasingly being integrated within tools to help perform rapid and automated computation on document sets. Yet, when taking advantage of these technologies, designers must be mindful of human factors when generating the user-facing interfaces of their tools, as a task cannot be performed effectively without direction from an empowered user.

Machine learning

Machine learning techniques are increasingly being utilized to tackle analytic problems once considered too complex to solve in an effective and timely manner. Yet, recent analysis on the human factors in machine learning environments have found that the current design strategies continually limit users’ ability to take part in the analytic process. More so, it has produced a generation of machine learning-integrated tools that are failing to provide users a complete understanding on how computational systems of their tools arrive at their results. This has significantly reduced users’ control and lowered the ability to achieve task objectives. In response, there is a growing desire to promote the “human-in-the-loop,” bringing the benefits of human reasoning back to the forefront of the design process.

When considering the interaction loop of a machine learning-integrated tool, Sacha et al. present a five-stage conceptual framework: producing and accessing data, preparing data for tool use, selecting a machine learning model, visualizing computation in the tool interface, and users applying analytic reasoning to validate and direct further use. Assessing this framework, a machine learning-integrated search tool must provide users with a functional workflow where:

1. users communicate their task requirements as a query;
2. users ask their tool to apply that query as input within its computational system;
3. the tool performs its computation, mapping the features against the document set;
4. the tool represents the results of the computation in its interface;
5. users assess whether they are or are not satisfied with the results; and
6. users restart the interaction loop with adjustments or conclude their use of the tool.

Thus, a primary responsibility for users within machine learning environments is the need to assess how well the results of machine learning have aligned with their task objectives. A systematic review by Amershi et al. suggests six considerations for the user’s role in arbitrating machine learning performance:

• Users are people, not oracles (i.e., they should not be expected to repeatedly answer whether a model is right or wrong).
• People tend to give more positive than negative feedback.
• People need a demonstration of how machine learning should behave.
• People naturally want to provide more than just data labels.
• People value transparency.
• Transparency can help people provide better labels.

Ontologies

In search tasks involving large document sets, many challenges can arise that reduce performance quality, harm user satisfaction, and increase the time for task completion. Often, these challenges result from misalignment between the vocabularies used by the document sets, storage maintainers, interface designers, and users. For instance, Qing et al. outline the difficulties faced when translating between common and domain vocabularies in health tasks. They describe a study that found that up to 50% of health expressions by consumers were not represented by public health vocabularies.

Within the pipeline of a search task, both the human and computational system can only perform optimally if communication is strong. Ontologies are representational artifacts that reflect the entities, relations, and structures of its domain. Ontologies are of three types: a philosophical ontology for describing and structuring reality, a domain ontology for structuring the entities and relations of a knowledge base, and a top-level ontology for interfacing between different domain ontologies. Ontologies provide the flexibility, extensibility, generality, and expressiveness necessary to bridge the gap when mapping domain knowledge for effective computer-facing and human-facing use. For this purpose, ontologies are increasingly being used within tools to help users perform their challenging search tasks.

When creating an ontology, experts construct a network of entities and relations, which together yield various structures. Ontology entities reflect the objects of the domain, like a phenotype in a medical abnormality ontology, a processor in a computer architecture ontology, or a precedent in a legal ontology. In some ontologies, like the top-level ontology, Basic Formal Ontology, designers go as far as denoting qualities such as materiality, object composition, and spatial qualities in reality. Ontology entities are encoded with information about their role in the vocabulary, definitions, descriptions, and contexts, as well as metadata that can inform the performance of future ontology engineering tasks.

Ontology relations are the links between ontology entities that express the quality of interaction between them and the domain as a whole. When assessing ontology relations, Arp et al. distinguish relations under the categories of universal–universal (dog “is_a” animal), particular–universal (this dog “instance_of” dog), and particular–particular (this dog “continuant_parts” of this dog grouping). Domain ontology relations are realized through unique interoperability between ontology entities. For instance, an animal ontology may have an ontology entity reflecting the concept of a “human,” which may have the ontology relations “domesticates/is domesticated by” between it and the “dog” ontology entity.

After defining the entities, relations, and other features of an ontology, experts record their work in ontology files of standardized data formats like RDF, OWL, and OBO. These ontology files are then distributed amongst users. They can then be integrated into the computational and human-facing systems of tools for use during tasks. Some examples of current ontology use are information extraction on unstructured text, behavior modeling of intellectual agents, and an increasing number of human-facing visualization tasks such as decision support systems within critical care environments.

Methods

In this section, we describe the methods used for criteria formulation. We begin with a review of literature for health informatics search tasks. Based on the insights gained from this review, we distill a set of criteria. We then use these criteria to contrast traditional design strategies for interfaces of search tasks.

Task review

Here, we review some research on interfaces for health informatics search tasks. We used Google Scholar, IEEE Xplore, and PubMed to conduct an exhaustive search of articles and reviews published between 2015 and 2021. We have divided our findings into three sections. First, we explore research on health data, information management, and information-centric interfaces. This is followed by research discussing the types of search tasks and their use in structuring the design of interfaces for health informatics. Finally, we investigate the requirements for aligning vocabularies for health informatics search tasks.

Health data, information management, and information-centric interfaces

Health data is constantly generated, highlighted by reports that within just a year the U.S. healthcare system created 150 new exabytes of data. Yet, the information that is expressed by this data, such as personal medical records, research publications, and consumer health media, is not useful unless it can be effectively understood and utilized by users. As such, it is critical to examine the challenges facing users when performing their tasks, and through this establish novel strategies for supporting the activation of health data.

Fang et al. explore the pressing challenges for accessing health data under the four categories: volume, variety, velocity, and veracity. They find that the volume of health care data creates challenges in the management of data sources and stores. They describe that existing strategies are struggling, and that novel designs should be established for scaling data services. They explain that a variety of challenges come with the management of data characteristics, ranging from unstructured datapoints generated from sources like sensors, to structured data entities like research papers and medical documents. For this, they state that designers should concentrate on aligning with the characteristics of the information being encountered. Next, they explore the challenges of velocity, which involves the rate at which users require their data to move from source to activation within their task. They highlight novel research in the networking and data management space. Finally, they explore veracity challenges, such as the assessment and validation of data quality and the quality of information that the data may produce.

Gibson et al. provide a review of the evolving fields of health information management and informatics. They review the topics of data capture, digital e-record systems, aggregate health management, healthcare funding models, data-oriented evidence-based medicine, consumer health applications, health governance, personal health access, and genomic personalized medicine. Similar to Fang et al., they note that the predominant work for health informatics should be concerned with presenting users with information-centric perspectives during their performances rather than technology-centered perspectives. More so, they describe that users in healthcare “must often navigate and understand complex clinical workflows to effectively … capture, store, or exchange information.” In other words, task workflows are already complex; therefore, effective interfaces should promote information encounters that help users perform better, rather than engage in unrelated technical details.

From the above research, we distill the criterion: Designs should maintain an information-centric interface that is flexible with respect to the dynamic requirements of search tasks like veracity of data sources, variety of data types, and evolving needs of users for health informatics.

Search tasks and structuring the design of interfaces for health informatics

Russell-Rose et al. describe professional health workplace tasks. They find that the most prevalent types of search tasks are literature reviews for overviewing a topic, scoping reviews for rapidly inspecting the possible relevance of an information source, rapid evidence reviews for appraising the overall quality of a scoping review, and, finally, systematic reviews for exploring a topic in a robust manner.

During search tasks, users often lack the ability to perceive how their query decisions impact, relate, and interact with the document set. This is an important consideration for users who might want to adjust a query to better align with their information-seeking objectives. A further study by Russell-Rose et al. analyzes search strategies performed by healthcare professionals. They find that a large majority of participants have a general desire to utilize advanced search functionalities when available. This suggests that users are not hesitant to take advantage of resources that they believe help optimize their task performance. Huurdeman [39] outlines that for this, a good course of action is to leverage query corrections, autocomplete, and suggestions. Yet, they find that such additions can be harmful if those features do not provide appropriate domain context. That is, resources must allow users to be contextually aware of how their query aligns with the contents of document sets, as well as the conditions of computational technologies used by interfaces.

In the same research, Huurdeman investigates complex tasks involving information search and information-seeking models when using multistage search systems. In this research, they explore requirements that designers must account for when supporting users. Challenging search tasks require users to learn about the searched domain, understand how their objectives align, and formulate their objectives into a way that can be used by their tool. In other words, query building requires users to be domain cognizant, as they must communicate information-seeking objectives in a way that is understood by the tool, yet also aligns with the information found within the document sets. Thus, a health informatics tool that supports search tasks should provide the opportunity for understanding the domain of the document set being explored.

Zahabi et al. describe a set of nine requirements for designers when considering how to design usable interfaces for health informatics search tasks, summarized as:

• Naturalness: The workflow of the system must present a natural task progression.
• Consistency: The parts of the system should present similar functional language.
• Prevent errors: Be proactive in the prevention of potential errors.
• Minimizing cognitive load: Align cognitive load to the requirements of the task.
• Efficient interaction: Be efficient in the number of steps to complete a task.
• Forgiveness and feedback: Supply proper and prompt feedback opportunities.
• Effective use of language: Promote clear and understandable communication.
• Effective information presentation: Align with information characteristics.
• Customizability/flexibility: The system should remain flexible to the task requirements.

Additional research by Dudley et al. reviews user interface design for machine learning environments. They provide a set of principles that can be used by designers:

• Make task goals and constraints explicit.
• Support user understanding of model uncertainty and confidence.
• Capture intent rather than input.
• Provide effective data representations.
• Exploit interactivity and promote rich interactions.
• Engage the user.

From this research, we can distill two criteria. First, designs should provide interaction loops that promote prompt and effective feedback opportunities for the user. Second, designs should provide representations that are natural and consistent to the requirements of the information source, the user, and the task.

Aligning vocabularies for health informatics search tasks

When considering interfaces for health informatics search tasks, a major challenge for users is the need to overcome problem formulation deficiencies when encountering unfamiliar domains. This is because, according to Harvey et al., users have been found to consistently suffer from four major issues during the performance of search tasks:

• difficulty understanding the domain being searched;
• an inability to apply their domain expertise;
• lacking the capacity to formulate an effective search query within the interface that accurately reflects their information-seeking objective; and
• deficient understanding of how to assess results produced by search, to decide whether the search has or has not satisfied their objective.

Harvey et al. show that in domains with complex vocabularies, such as health and medicine, the disparity of potential users’ prior knowledge is extreme. They find that non-expert users routinely do not possess enough domain knowledge to address their information-seeking needs. This can cause significant issues during query formulation. As a result, non-expert users must first step away from their tool to learn specialized vocabulary before they can begin query building. Both Soldaini et al. and Anderson and Wischgoll describe that this issue can still affect even experts. This is because experts often must make assumptions when attuning to their tool.

There is growing research targeting the generation and application of mediation resources to help reduce the communication gap while using health informatics tools. Zeng et al. investigate the development of consumer health vocabularies for reducing the discourse gap between lay people and medical information document sets. Furthermore, Soldaini et al. explore the use of novel query computation strategies to improve the quality of medical literature retrieval during search tasks. In their quantitative study, they contrast models generated using combinations of algorithms, vocabularies, and feature weights, assessing the computational performance of different query reformulation techniques. The results of their study suggest “greatly improved retrieval performance” when utilizing combined machine learning and bridged vocabularies. More so, they provide insight regarding the quality of options that can support computational systems for health informatics search tasks.

From the above research, we can distill two criteria. First, designs should provide interactions that allow users to efficiently prepare, perform, assess, and adjust their machine learning to align with information-seeking objectives of search tasks. Second, designs should provide mediation opportunities that assist users in communicating information-seeking objectives into the domain-specific vocabulary of the document set.

High-level criteria

In Table 1, we provide five criteria based on the above review.

Table 1. The criteria for guiding the design of interfaces for health informatics search tasks involving large document sets. For abbreviation purposes, design criteria will be referenced in the text as DC#, where # is its assigned number. DC# Design criteria DC1 Provide an information-centric interface that shows flexibility towards the evolving needs of users and the dynamic requirements of search tasks like the veracity of data sources and variety of information types. DC2 Provide interaction loops that supply prompt and effective feedback for users during the performance of search tasks. DC3 Provide natural and consistent representations that allow users to understand the constraints, processes, and results provided by the interface. DC4 Provide interactions that allow users to efficiently prepare, perform, assess, and adjust their machine learning to align with the information-seeking objectives of search tasks. DC5 Provide mediation opportunities that assist users in communicating and bridge their information-seeking objectives into the vocabulary of the document set.

Analysis of traditional interface strategies for health informatics search tasks

We now assess the traditional design strategies for interfaces for health informatics search tasks. Wilson’s comprehensive Search User Interface Design provides a complete survey of the history and current state of search interfaces. Based on their survey, and in particular their discussion of input and control features within the modern search user interfaces, two base strategies and one extension strategy for search interfaces are realized: “structured” interfaces, “unstructured” interfaces, and, in extension, “query expansion” interfaces. Table 2 provides a summary of how the above criteria align with each interface strategy.

Table 2. A summary matrix of alignment between the criteria and interface strategies. Full descriptions are found within their respective sections. “Strong” is assigned if a characteristic of the interface strategy promotes alignment with the requirements of the design criterion. “Weak” is assigned if a characteristic of the strategy does not promote alignment with the requirements of the design criterion. “Variable” is assigned if the interface strategy has the potential to align with the criterion; however, such an alignment is not innate and must be actively pursued. DC# Structured Unstructured Query expansion DC1 Weak Variable Strong DC2 Strong Strong Variable DC3 Variable Variable Variable DC4 Variable Strong Strong DC5 Weak Weak Strong

Notes

This presentation is faithful to the original, with only a few minor changes to presentation. Some grammar and punctuation was cleaned up to improve readability. In some cases important information was missing from the references, and that information was added.