Journal:Avoidance of operational sampling errors in drinking water analysis

Full article title	Avoidance of operational sampling errors in drinking water analysis
Journal	AQUA - Water infrastructure, Ecosystems and Society
Author(s)	Fernandes, Ana; Figueiredo, Margarida; Ribeiro, Jorge; Neves, José; Vicente, Henrique
Author affiliation(s)	Universidade Lusófona, Universidade de Évora, Instituto Politécnico de Viana do Castelo, Instituto Universitário de Ciências da Saúde, Universidade do Minho
Primary contact	Email: hvicente at uevora dot pt
Year published	2022
Volume and issue	71(3)
Page(s)	373–386
DOI	10.2166/aqua.2022.074
ISSN	2709-8036
Distribution license	Creative Commons Attribution 4.0 International
Website	https://iwaponline.com/aqua/article/71/3/373/87050
Download	https://iwaponline.com/aqua/article-pdf/71/3/373/1026718/jws0710373.pdf (PDF)

Abstract

The internal audits carried out in the first half of 2019 in water laboratories as part of quality accreditation in accordance with ISO/IEC 17025:2017 showed a high frequency of adverse events in connection with sampling. These faults can be a consequence of a wide range of causes, and in some cases, the information about them can be insufficient or unclear. Considering that sampling has a major influence on the quality of the analytical results provided by water laboratories, this work presents a system for reporting and learning from adverse events. Its aim is to record nonconformities, errors, and adverse events, making possible automatic data analysis to better ensure continuous improvement in operational sampling. The system is based on the Eindhoven Classification Model and enables automatic data analysis and reporting to identify the main causes of failure. Logic programming is used to represent knowledge and support the reasoning mechanisms to model the universe of discourse in scenarios of incomplete, contradicting, or even unknown information. In addition to suggesting solutions to the problem, the system provides formal evidence of the solutions presented, which will help to continuously improve drinking water quality and promote public health.

Keywords: drinking water, Eindhoven Classification Model, Knowledge Representation and Reasoning, logic programming, sampling errors, water quality, decision support system

Highlights:

• An adverse event reporting and learning system for water sampling is described.
• The Eindhoven Classification Model is extended and adapted for water sampling.
• Logic programming is used for knowledge representation.
• The proposed system can deal with insufficient or ambiguous information.
• The system allows users to identify the relevant issues behind the errors that may occur.

Introduction

Securing water quality for human consumption through a public supply system is an essential element of health policy. By the end of the nineteenth century, an assessment and control of risks to human health from transmission of diseases caused by water consumption had been carried out empirically on the basis of the physical appearance of water. [Bagchi 2013] Epidemiological studies conducted by John Snow showed a close association between the consumption of water with fecal contamination and a cholera outbreak in London. [Snow 1855] The discovery of the existence of microorganisms by Louis Pasteur in 1863 and the isolation of Vibrio cholera bacillus in 1883 by Robert Koch formed the crucial scientific basis for the association of water use with public health and served as a starting point for the establishment of practices and protocols to control water quality. [Geison 1995; Brock 1999]

Until the middle of the twentieth century, the quality of water for human consumption had been largely assessed based on its organoleptic characteristics, i.e., its colorless, tasteless, and odorless features. [Eaton et al. 2017] However, this type of assessment does not guarantee the protection of public health from pathogenic microorganisms and hazardous chemicals. Therefore, it is imperative to establish standards based on quantifiable parameters that define the properties of water for human consumption in terms of microbiological, physical, chemical, and radiological factors. [ISO/IEC 17025 2017] Current methods of controlling water quality are time-consuming and complex, and they involve the use of significant technical and financial resources (e.g., calibrated equipment, infrastructure, skilled technicians, and reagents). With all this complexity, a large number of errors and failures can occur. [Byleveld et al. 2008] A failure can be defined as a planned action not meetings its required end result, at times because of use of an incorrect plan. These failures can be in connection with products, processes, and systems. [Hommerson et al. 2008]

The best way to prevent similar errors and failures from happening again is to report them, i.e., create experiential learning systems to identify their causes. Although this approach is widely used in the water sector, it is important to note that classification systems work better when they are limited to a specific field/stage/phase (e.g., pre-analytical, analytical, and post-analytical phases). Sampling is an important stage in a water quality control program as it can affect results if a sample is not representative of the water being controlled. On the one hand, sampling errors can arise if the sampling is not designed to answer specific questions (usually regulators specify the sampling approach and the sampling design to be adopted). On the other hand, sampling errors can also be related to operational issues.

In this work, a system for reporting and learning from errors is presented, with the aim of describing and preventing those that occur in operational sampling of water for human consumption. The operational sampling of water for human consumption involves four main steps: collection, conditioning, preservation, and transportation. These steps are depicted in Figure 1. Water samples are delivered to the laboratory for analysis, and the sampling technician is responsible for taking a valid and representative sample. [Li & Migliaccio 2011]

Figure 1. The main steps in operational sampling of water for human consumption.

Due to the increasing importance attached to the representativeness of sampling, greater emphasis has been placed on the adequate collection, conditioning, preservation, and transport of samples. Thus, it is advisable that laboratories help to plan a sampling program in partnership with their customers. This partnership is essential to ensure that the selection of samples is adequate in order to minimize errors and to correspond to the needs of the consumer. [Nollet & Gelder 2014] At this point, it is important to reiterate that the main goal of water analysis is to aid the decision-making process.

Thus, the reliability of the decisions made depends on the variability of the experimental results. Despite variability being inherent to natural systems, if the variability due to errors in sampling, transport, handling, and/or analysis is underestimated, wrong decisions can be made, with negative consequences for health, environment, and financial matters, just to name a few. Indeed, the errors that occur during the water analysis process are difficult to quantify. Inadequate training, over-reliance on automated systems, and complicated software resources are some of the relevant issues behind the errors that may occur. [Reason 2000] More broadly, knowledge is crucial to solve the problems of the economy and society. In everyday situations, however, the information that flows from various sources is often incomplete, inaccurate, uncertain, contradictory, and even unknown. On the other hand, existing models are built based on some idealizations that remove these real-world properties.

The result is a system that, due to its inability to model the world or universe of discourse, never delivers the expected answers. [Parsons 1996] In the water sector, particularly with regard to sampling errors, there are many situations where the information available on errors is insufficient or ambiguous. The system proposed in this work is based on the Eindhoven Classification Model (ECM) [van der Schaaf 1995] and focuses on avoiding the causes of errors by applying a model specially developed for the operational sampling phase.

The ECM was created to address human error in the chemical process industry and has since been applied in different areas. [Henneman et al. 2006; Raab et al. 2006; Simmons & Graves 2008; Rodrigues et al. 2011; Vicente et al. 2015; Fernandes et al. 2019] The ECM classifies two types of errors, namely, human or active errors and latent ones. With regard to human error, the ECM incorporates Rasmussen's SRK thematic, which includes three levels of behavior, namely, competence-based, rule-based, and knowledge-based behavior levels. [Rasmussen 1976] In the case of latent errors, the ECM differentiates between technical and organizational errors [van der Schaaf 1995] The former occurs when there is an obstacle associated with physical components (e.g., devices and physical installations), while the latter is associated with protocols, procedures, or knowledge transfers.

The main objective of this work is to present an adverse event reporting and learning system, developed specifically for the sampling phase. Based on the internal audits carried out in the first half of 2019 in Portuguese water laboratories, the adverse event "Failure in Sampling" was chosen to demonstrate how the proposed system can be applied to real-world problems. This system, known as Adverse Event Reporting and Learning System related to Water Sampling (AERLS-WS), intends to record the nonconformities, errors, and adverse events that occur during sampling. It also intends to be a learning system that enables data analysis to ensure continuous improvement in drinking water quality and the promotion of public health.

Related work

The need for accurate predictions of water quality has encouraged researchers to develop new methods and improve the predictive capabilities of traditional ones. This includes artificial intelligence (AI)–based problem-solving methods, which have seen adoption in the water industry. Several studies in the literature illustrate the application of these tools for quality water control, but very few relate to drinking water. Indeed, water quality modeling is a powerful tool for studying, describing, and predicting the ecological status of an aquatic ecosystem. Morales-Torres et al. [2016] presented a study about the use of sustainable drainage systems to manage urban runoff, aiming to contribute to environmental and landscape improvement. The authors developed a tool to analyze the impact of storm water management on the urban environment and introduced energetic and environmental criteria in the decision-making process. Stefanovic et al. [2015] developed a web-based system to manage Serbian reservoirs. The authors created a software that brings together all the relevant actors, processes, data sources, and applications. The software allows for testing different organizational scenarios, due to its flexibility, extensibility, and the fact that it could be customized. Allan et al. [2013] studied the effects of regulation, proprietorship, and business culture on risk management within the water sector. The authors stressed that a paradigm shift is necessary. The water sector should move from a reactive to a pro-active risk management culture based on knowledge. However, the authors stated that there is a gap in the understanding of the drivers required to bring about such a change. Pearson et al. [2010] presented a decision support system to support managers in the urban water sector. The proposed system aims to shift decision-making from the conventional standards to a sustainable, inclusive, and dynamic paradigm, anchored by social learning and commitment. The authors stressed the value of the basic units of social learning, i.e., knowledge transfer for transformation, monitoring and evaluation of the decision-making process, and stakeholder engagement.

In the water sector, there are many situations in which information is incomplete. However, most of the decision support systems ignore this fact. Most of the time, such systems do not provide the expected answers due to their inability to represent the universe of discourse. Thus, it is of utmost importance to deal with defective information. In this context, the use of the logic programming (LP) paradigm to express theories has grown in recent years, and many other fields have started using its tools and methodologies. Theories are typically expressed as a succession of rules and facts that allow to deduce non-logical consequences using logical inferences. [Fernandes et al. 2020] According to the extension to the LP language presented in Neves [1984], both explicit (or strong) negation and explicit declarations may be used in the establishment of rules and facts. Furthermore, theories can be enhanced by adding special rules in the form of invariants to specify the context under which the universe of discourse should be understood. Such invariants specify the conditions that must be met and must have overall consistency, i.e., it must not be possible to infer a conclusion and its contradiction. [Neves 1984; Kakas et al. 1998; Neves et al. 2021] This extension to the LP language uses the open-world assumption (OWA), i.e., everything is unknown or undefined until one has some secure evidence of its truthfulness or falseness. [Keet 2013] The OWA differs from the more usual closed-world assumption (CWA), where everything is assumed false until there is solid evidence of its truthfulness. [Keet 2013] The use of the OWA is the main advantage of the method used, since it allows to deal with incomplete, inconsistent, contradictory, default, or forbidden information, and the conclusions are an object of a formal proof. Indeed, this point is of utmost importance, since in the real world, information is incomplete and updateable.

Methods

The ECM was initially conceived to address errors in the chemical industry [van der Schaaf 1995] and has since been adapted to be used in different areas, e.g., the steel industry, power generation, healthcare, water distribution, or public administration. [Henneman et al. 2006; Raab et al. 2006; Simmons & Graves 2008; Rodrigues et al. 2011; Vicente et al. 2015; Fernandes et al. 2019] In this work, an adjusted version of the ECM was conceived, with extensions and adaptations for water sampling scenarios, as well as causal trees (CTs) that describe the root causes of errors. The data used in this work were obtained during the first semester of 2019, in internal audits carried out at the Water Laboratory of Santiago do Cacém Municipality (located in Alentejo Litoral, southern Portugal), within the scope of ISO/IEC 17025:2017 quality accreditation. Checklists were used to record nonconformities. In the checklists, verified observations were organized according to the general and specific requirements of the ISO/IEC 17025 standard. In this work, the nonconformities were classified into major nonconformity, minor nonconformity, opportunity for improvement, and in terms of the ECM, referred to as adverse events or errors.

The formal approach of the proposed errors reporting system in water sampling is based on the LP paradigm for problem-solving and is set according to an extension to the LP language. [Neves 1984] Undeniably, there are many approaches to knowledge representation and reasoning (KRR) using the epitome of LP, namely, in the areas of model theory and proof theory. In this article, the Proof Theoretical approach for problem-solving was adopted and conveyed as an extension of the LP language. Indeed, an extended logic program (ELP) is grounded on a specified ensemble of clauses in the form of Program 1:

Program 1. The archetype of an extended logic program

...where p_i, q_j, and p stand for a set of predicates. A predicate is the part of a sentence that contains the verb and tells something about the subject. In Program 1, "," designates "logical and" while "?" denotes "falsity." The classical negation sign "⌝" denotes a strong declaration, whereas "not" denotes negation by failure, i.e., a collapse in demonstrating a certain statement, since it is not declared in an explicit way. [Neves 1984] According to this way of thinking, a set of exceptions are presented in every program [Neves 1984], i.e., clauses in the form:

$\displaystyle {e}{x}{c}{e}{p}{t}{i}{o}{n}_{{p}_{1}},\ldots ,{e}{x}{c}{e}{p}{t}{i}{o}{n}_{{p}_{j}}{\left({0}\leq {j}\leq {k}\right)},{\text{being k an integer number}}$

...that denotes data, information, or knowledge that cannot be ruled out. On the other hand, clauses of the type:

$\displaystyle ?{\left({p}_{1},\ldots ,{p}_{n},{\text{not }}\ {q}_{1},\ldots ,{\text{not }}\ {q}_{m}\right)}{\left({n},{m}\geq {0}\right)}$

...are invariants that make it possible to specify the context under which the universe of discourse should be understood. [Neves 1984]

In order to isolate a customized version of the ECM for a particular problem, it is of the utmost importance to identify all the causes that lead to such an undesirable event. To adapt the ECM to the problems related to the sampling of water for human consumption, new dimensions have been considered. These dimensions aim to adjust the original model to the sampling context and provide a general view of the events that can happen. The Adjusted version of ECM for Water Sampling (A-ECM-WS) followed the dimensions "technical," "organizational," "human behavior," and "other," presented in Table 1.

Table 1. Categories of the adjusted ECM for water sampling and the corresponding codes. Category Description Code Technical dimension External Technical faults outside the control and responsibility of the company TEX Design Faults caused by inadequate design of equipment, software, tags, or template TD Construction Construction faults regardless of a right design TC Materials Material deficiencies not categorized as TD or TC TM Reagents Faults caused by alack or poor quality of chemicals TR Organizational dimension External Faults at an organizational level outside the control of the company OEX Transfer of knowledge Faults due to inadequate decisions that do not guarantee that the knowledge is shared with inexperienced employees OK Protocols Faults due to the quality/availability of the internal procedures (excessively intricate/simple, imprecise, or non-existent) OP Management priorities Internal choices in which safety is transferred to a lower position, giving priority to production OM Culture Faults due to the collective attitude towards risk and behaviors within the company OC Human behavior dimension External Human faults that have its origins outside the control of the company HEX Knowledge-based behavior Knowledge-based errors The disability of a member to put in practice current knowledge in a new situation HKK Rule-based behavior Qualifications Inappropriate fit among the task to be performed and the employee's training, qualifications, or education HRQ Coordination Deficient task coordination (e.g., a fundamental task not fulfilled because the team was convinced that someone performed it) HRC Verification Faults during the performance of an accurate and full assessment of the set of circumstances before starting the execution of a specific task, covering the conditions and materials and/or equipment to be used HRV Intervention Faults that arise from defective preparation of a task and/or bad implementation HRI Monitoring Faults that occur during the monitoring of an activity or process HRM Skill-based behavior Slips Faults in performing a task related to the operator's lack of fine motor skills HSS Other dimension Technicians' related factor Faults associated with physical and/or mental conditions of the employee that affect the task execution and are outside the control of the company TRF Consumers' related factor Faults associated with consumers’ conduct that can affect water quality and are outside the control of the company CRF Unclassifiable Faults that cannot be included in the previous categories X

Table 1 also shows the categories included in each dimension, as well as a brief description and the corresponding codes. To illustrate the type of adverse events that can be included in each code, Table 2 presents a few examples. Thus, based on Tables 1 and 2, the systematization of the causes that lead to an adverse event becomes straightforward and effective, being processed according to the flow chart presented in Figure 2. For instance, an adverse event assigned to the category "Organizational dimension – Protocols" (OP) occurs due to the quality/availability of the internal procedures. The events assigned to this category arise from difficulties in execution or interpretation of unclear protocols that can lead to adverse events such as the failure of not flowing the stagnant water in the faucet or the failure to remove the faucet accessories.

Table 2. An excerpt from the code set of the adjusted ECM for water sampling and examples of possible adverse events. Code Extension to water sampling Examples TR Faults caused by a lack or poor quality of chemicals Unintended use of out-of-date reagents OP Very complicated, inadequate, or missing protocols, or protocols not disclosed to the team Allowed nonconformity of protocols with various negative effects (e.g., non-flow of stagnant water in the duct section, failing to remove accessories outside the tap) HEX Human faults outside the control of the company Allowed excessive time between sampling and delivery to the laboratory HKK Failure to put on preservatives or choosing the wrong preservative Failed to add HNO3 in the samples for determining iron and manganese HRI Failure in task planning or during its execution Did not disinfect hands before sampling, did not use gloves, did not use the blowtorch correctly, and did not choose a tap under normal conditions of conservation and hygiene HRM Faults that occurred during the monitoring of a task Failed to calibrate the data logger used inside the thermal bags

Figure 2. The adverse event classification process.

The ECM makes use of root cause analysis to classify adverse events through codes established in advance. [van der Schaaf 1995] Thus, beyond the classification framework presented above, CTs regarding each adverse event should be conceived. CTs can be considered as a hierarchical representation of an adverse event, allowing to identify the circumstances that lead to its occurrence. [van der Schaaf & Habraken 2005] The original version of the ECM only comprises cases for which the causes of adverse events are well known. This fact should be a limitation, since in many situations information about errors and adverse events can be incomplete, unknown, or even contradictory. To include the uncertainty that is always present in real situations, in the present work, the unknown and forbidden operators were considered in the conception of the extended causal trees (ECTs).

Application

Table 3 lists the adverse events and the number of occurrences recorded in the internal audits carried out in the Water Laboratory of Santiago do Cacém Municipality (located in Alentejo Litoral, southern Portugal) during the first semester of 2019. A perusal of Table 3 reveals that 128 adverse events occurred. Considering the general requirements of the ISO/IEC 17025 standard, 28.9% of the occurrences were related resource requirements, 53.9% were associated with processes requirements, and 17.2% were linked to management system requirements. In terms of nonconformities, 81.2% were classified as a minor nonconformity (N) and 18.8% corresponded to a major nonconformity (M).

Table 3. Number of adverse events that occurred during the first semester of 2019 at the Water Laboratory of Santiago do Cacém Municipality. M = major nonconformity; N = minor nonconformity. General requirements Specific requirements Description Classification Number of occurrences Total Resource requirements Facilities and environmental conditions Laboratory facilities are not adequate (e.g., no separation between incompatible testing areas). M 2 7 The laboratory did not formalize the facilities and environmental conditions. N 5 Metrological traceability The laboratory did not formalize the methodology for estimating uncertainties. N 7 22 The calculation of the uncertainty estimate for the temperature test was not shown. N 9 The calculation of the uncertainty estimate for free and total residual chlorine was not shown. N 6 Equipment The laboratory did not show records of annual visual inspections of the volumetric glass material. N 8 8 Process requirements Nonconforming work The methodology for nonconforming work is not formalized. M 10 10 Selection, verification, and validation of methods The laboratory has not formalized the requirement methods/procedures for testing and validating the methods. N 5 9 The laboratory did not formalize the methodology for the implementation of the test methods. N 4 Sampling Sampling technician failures were found (e.g., failure in disinfection). N 12 50 It was found that in the procedure there were deficiencies in the location of the sampling point. N 6 It was found that inappropriate bottles were being used to carry out the sampling. N 12 Failures were found in the preservation of samples. N 10 Transport failures were noted (e.g., agitation during transport, time between collection, transport, and testing). N 10 Management system requirements Documentation Documentation control is neither adequate nor formalized. M 12 22 The external documents to be controlled are not defined. N 10 Adverse events registered 128

Notes

This presentation is faithful to the original, with only a few minor changes to presentation. Grammar was tweaked significantly to improve readability. The PMCID and DOI were also added when they were missing from the original reference. The original article lists references alphabetically, but this version—by design—lists them in order of appearance.