Journal:Digital transformation risk management in forensic science laboratories

Full article title	Digital transformation risk management in forensic science laboratories
Journal	Forensic Science International
Author(s)	Casey, Eoghan; Souvignet, Thomas R.
Author affiliation(s)	University of Lausanne
Primary contact	Email: thomas dot souvignet at unil dot ch
Year published	2020
Volume and issue	316
Article #	110486
DOI	10.1016/j.forsciint.2020.110486
ISSN	0379-0738
Distribution license	Creative Commons Attribution 4.0 International
Website	https://www.sciencedirect.com/science/article/pii/S0379073820303480
Download	https://www.sciencedirect.com/science/article/pii/S0379073820303480/pdfft (PDF)

Abstract

Technological advances are changing how forensic laboratories operate in all forensic disciplines, not only digital. Computers support workflow management and enable evidence analysis (physical and digital), while new technology enables previously unavailable forensic capabilities. Used properly, the integration of digital systems supports greater efficiency and reproducibility, and drives digital transformation of forensic laboratories. However, without the necessary preparations, these digital transformations can undermine the core principles and processes of forensic laboratories. Forensic preparedness concentrating on digital data reduces the cost and operational disruption of responding to various kinds of problems, including misplaced exhibits, allegations of employee misconduct, disclosure requirements, and information security breaches.

This work gives pertinent examples of problems and risks involving technology that have occurred in forensic laboratories, along with opportunities and risk mitigation strategies, based on the authors’ experiences. It also presents recommendations to help forensic laboratories prepare for and manage these risks, to use technology effectively, and ultimately strengthen forensic science. The importance of involving digital forensic expertise in risk management of digital transformations in laboratories is emphasized. Forensic laboratories that do not adopt forensic digital preparedness will produce results based on digital data and processes that cannot be verified independently, leaving them vulnerable to challenge. The recommendations in this work could enhance international standards such as ISO/IEC 17025, which are used to assess and accredit laboratories.

Keywords: forensic science, digital transformations, forensic laboratories, forensic preparedness, forensic digital preparedness, risk management, ISO/IEC 17025

Introduction

Forensic science laboratories are becoming more reliant on computers and data for both administrative and analytical operations. These technological advances create new opportunities and risks for all forensic disciplines, not only to digital evidence. With proper preparation and management, forensic laboratories can employ technology effectively to improve performance and quality, while mitigating the associated risks. However, many forensic laboratories do not understand the subtlety and expertise required to manage risks of digital transformation, inadvisedly treating it as simply a technical component of existing quality management processes. Forensic laboratories that fail to realize the need for forensic digital preparedness to actively manage risks associated with digital transformations are vulnerable to significant expense, disruption, and liability when problems arise.

Forensic laboratories rely on technology for much more than communication and routine business functions. Sophisticated equipment for processing chemical and biological materials are operated using computers and save results in digital form. Mass spectrometers, DNA analysis systems, and other laboratory equipment save their results in raw data files. Digital evidence is processed using specialized hardware and software, although not all forensic laboratories have integrated this new discipline. Forensic laboratories are using computerized case management systems for tracking treatment of all evidential exhibits and forensic results. Automated systems with artificial intelligence (AI) are being used to support forensic analysis. In reality, digital transformations—the use of digital technology to make existing processes more efficient and effective, and to develop new solutions to emerging problems—are well underway, and forensic laboratories require a robust strategy to manage the associated risks and realize the opportunities.

This increased dependence on digital technology creates risks and opportunities for forensic laboratories. Potential pitfalls include loss of data needed to perform forensic analysis, errors in analysis of physical traces (e.g., DNA, fingerprint, face) caused by computer hardware or software, ability to tamper with raw data files generated by laboratory equipment, and incorrect information input into laboratory information management systems (LIMS). Possible benefits are traceability and integrity of traces, reliability and reproducibility of results from information extracted from traces and stored as raw data, and use of AI to support forensic analysis.

Lessons can be learned from the digital forensic domain, including forensic digital preparedness and accreditation challenges. Primary challenges encountered by digital forensic laboratories adopting quality standards include:

• Inaccurate or insufficient information in technical records, including chain of custody, and no mechanism to detect subsequent changes to records.
• Problems with the security of information technology systems and the backup processes of data.
• Missing or insufficiently detailed procedures for treating digital data, and personnel not following documented procedures consistently.
• Lack of robust quality checking mechanisms, and issues with validation of methods.

This paper presents risks and opportunities associated with digital transformation of forensic laboratories, providing examples based on the authors’ experiences. Examples have been anonymized, as the intention is to illustrate general lessons learned rather than critique specific laboratories. This work then presents forensic digital preparedness, a set of recommendations to help laboratories navigate risks associated with digital transformations, including mishandled exhibits, allegations of employee misconduct, and disclosure requirements. The role of digital forensic capabilities and expertise in risk management of digital transformations in laboratories is discussed. This work culminates with broader implications for international standards such as ISO/IEC 17025, which are used to assess and accredit laboratories.

Risks and remedies

Many processes in forensic laboratories have become digitalized through the increased use of information management systems and software running analysis instruments. While these systems serve crucial functions in modern forensic laboratories, thet also have associated risks that must be managed.

Data retention

The computer systems used to store instruments' generated data files (raw and processed) can encounter problems that lead to loss of information.

Data loss scenario

In this scenario, Reust et al. presented a case study concerning a forensic laboratory that performed DNA analysis of a crime scene sample relevant to a multiple homicide and death penalty case, but did not retain a copy of the raw data files. To comply with a court order to provide the defense with original raw data, it was necessary to perform costly forensic data recovery on the computer used to perform the original processing of DNA. The authors developed a customized software utility to automatically search the computer hard drive for all fragments of the relevant raw data and reconstruct the original files. The resulting files were tested and validated with DNA analysis software.

As seen with Reust et al., original data files thought to be lost can, under certain circumstances, be recovered from hard disks using digital forensic methods, which can be costly and time-consuming. Even when digital data is retained, it is malleable and subject to undetected alterations of content or metadata. Lack of proper data retention processes makes it more difficult, sometimes impossible, to recover original data files and verify their integrity.

Generally, normal backup processes do not have the fidelity of digital forensic preservation mechanisms. To manage the risks of data loss and undetected alterations, traditional data retention practices in forensic laboratories can be updated to employ digital forensic preservation methods. Specifically, as part of routine data retention processes, digital forensic preservation of original data (raw and processed) and associated metadata (filesystem timestamps) allows the integrity of data to be verified more easily when there is a problem or inquiry. For instance, original files and associated metadata can be forensically preserved using the Advanced Forensic Format (AFF4), which is open-source and cross-platform. The following command and resulting output demonstrate how this method can be implemented on any type of computer system with a single command that can be part of a routine or automated process to forensically preserve all raw data files in a specified directory on a laboratory computer, while generating a unique identifier for the digital evidence container for evidence management purposes:

% aff4.py -cr s1-001-10April2020.aff4 RAWdata/s1-001

Creating AFF4Container: file://s1-001-10April2020.aff4

<aff4://c293153c-a317-4927-b1eb-6e3a5008ad0f>

Adding: RAWdata

Adding: RAWdata/s1-001/s1-001-sequence.sld

Adding: RAWdata/s1-001/s1-001-processed.pdf

Adding: RAWdata/s1-001/s1-001-ref.params

Adding: RAWdata/s1-001/s1-001.RAW

This digital forensic preservation process captures file system metadata and automatically computes MD5 and SHA1 cryptographic hash values of the acquired data for integrity verification purposes as the following excerpt shows:

% aff4.py -m s1-000-10April2020.aff4

... EDITED FOR BREVITY...

<aff4://c293153c-a317-4927-b1eb-6e3a5008ad0f/RAWdata/s1-001/s1-000.RAW>

a aff4:FileImage,

aff4:Image,

aff4:ImageStream;

aff4:birthTime “2020-04-10T22:41:03.949269+02:00”^◯sd:dateTime;

aff4:hash “1d2f7ff1ea563ceb6d2da0e168e90587”^âff4:MD5,

“427bc17e608fc493f0e2b3fed8fa55b36862ac31”^âff4:SHA1;

aff4:lastAccessed “2020-04-10T22:41:08.708498+02:00”^◯sd:dateTime;

aff4:lastWritten “2020-04-10T22:41:05.290019+02:00”^◯sd:dateTime;

aff4:originalFileName “RAWdata/s1-001/s1-000.RAW”^◯sd:string;

aff4:recordChanged “2020-04-10T22:41:07.694584+02:00”^◯sd:dateTime;

aff4:size 276196936.

These hash values are commonly used in digital forensic tools to enable future verification that the acquired data have not been altered since they were forensically preserved. The preserved metadata can also be useful for assessing the authenticity of the acquired data, including the original file name, size and creation timestamp.

Additionally, AFF4 assigns a unique identifier to the acquired data to support evidence management and provenance tracking.

Evidence integrity

The data files generated by laboratory equipment and stored on computers can be altered afterwards accidentally or intentionally.

Data alteration scenario

In this scenario, imagine data files stored on laboratory computers have been altered to conceal specific information in test results. Some alterations were detectable within the digital file, while others were not detected using available verification software. As a result, it was difficult to determine the full scope and specific impact of the alterations.

The motivation for editing the data files (raw and processed) might be to cover up mistakes, conceal unfavorable results (corruption), facilitate prosecution (bias), or inflate laboratory metrics (performance) Forensic laboratory personnel might modify data to remove traces of contamination they considered to be insignificant, such as traces of investigators operating an evidential smartphone after the device was seized. Depending on the type of data and the method of modification, it might be possible to detect the alteration. However, some alterations may be undetectable using existing verification tools, making it more difficult to determine that modifications were made.

Normal backup processes, and even digital forensic preservation such as described in the previous section using AFF4, are not tamperproof because data can be forged to replace retained data, and a computer system can be backdated to make it seem to have occurred sometime in the past. Lack of a tamperproof chain of custody of primary data sources in a forensic laboratory makes it more difficult, sometimes impossible, to authenticate original data files that form the basis of forensic findings and reported results.

To manage the risks of inadvertent alteration and intentional tampering, traditional provenance tracking practices in forensic laboratories must be updated to employ digitalized chain of custody ledger solutions. These digitalized chain of custody mechanisms can be implemented in a way that is tamperproof and independently verifiable.

Data traceability

Forensic laboratories are increasingly using a LIMS to record information about the full lifecycle of evidence in a forensic laboratory, including submission data, chain of custody, and results. A typical LIMS uses databases to store and organize information about each item of evidence at different stages of its treatment in the laboratory.

A LIMS is invaluable for keeping track of the growing amount of evidence and associated processes and results in forensic laboratories. As a result, such systems are considered essential for laboratory accreditation under standards such as ISO/IEC 17025. However, these systems can have weaknesses, including user-based data entry errors, programming bugs, and system administrator bypass of access controls.

LIMS weaknesses scenario

In this scenario, imagine that the results of drug tests have been routinely recorded in a LIMS, and normal users of the system can only create new records and view existing records. However, a system administrator was able to alter records using his higher level access, bypassing the security control mechanisms of a LIMS. As such, the LIMS maintained an audit log of all normal user activities, but did not log administrator-level actions.

To manage these risks of undetected or unattributed alterations to LIMS data, it is necessary to require unique user accounts for all actions and to maintain detailed electronic audit logs. These audit logs must include successful actions, not only failed or blocked actions. Specifically, all transactions must be recorded (additions, alterations, deletions), and all computer system usage, such as logons and executed commands. In particular, system administrator accounts should be strictly protected (e.g., via two-factor authentication) and monitored (e.g., via sudo logging and process accounting). All audit logs must be preserved in a forensically sound manner in anticipation of their use as digital evidence in a legal matter. Applying digital forensic preservation and digitalized chain of custody on logs generated by a LIMS and other supporting computer systems can be an efficient way to enhance LIMS traceability.

Computer system malfunction

Forensic laboratories increasingly depend on computers to operate equipment for extracting information from biological and chemical samples (Figure 1).

Figure 1. Generalized depiction of laboratory equipment creating digital output.

The computer systems used to operate laboratory equipment can malfunction, introducing errors in forensic analysis.

Hardware issues scenario

In this scenario, imagine, unbeknownst to administrators, a few DNA analysis systems in a forensic laboratory are operated by computers with slightly different hardware than the standard configuration. This seemingly minor difference inevitably caused read errors, which resulted in erroneous reference data being accessed on the DNA analysis systems. As a result, incorrect reference data were used in some cases, and the forensic analysis had to be repeated. This demonstrates that a seemingly unrelated problem with computer used to operate equipment for performing laboratory processes can cause incorrect results.

This exemplifies how seemingly minor changes to underlying computer systems can interfere with traditional forensic processes. Although validation of computer systems can be covered under existing laboratory management processes, the subtleties of computer hardware and software configurations and interactions must not be underestimated.

Automation complexity and pitfalls

In forensic contexts, use of automated systems, including those with AI and machine learning (ML), support analysis performed by human specialists who interpret the results. Although such automation can help maintain consistency and increase efficiency in forensic analysis, there are several major limitations that must be guarded against. Automated systems can have bugs that produce incorrect results, which can have serious consequences in a forensic context. Additionally, automated AI/ML systems can introduce bias due to poorly selected training datasets, and can lead to misinterpretations when the results are not fully understood. When automated AI/ML systems are used to support investigation and forensic analysis, such as comparison of faces in digital video or photographs, algorithmic false positives can lead to incorrect results.

Reliability and human rights concerns scenario

In this scenario, an independent study of six facial recognition technology test deployments performed by the London Metropolitan Police Service (MPS) found a high number of false positives. In total, only eight out of 46 automatically generated potential face recognition “matches” that were considered and evaluated by a human were deemed to be correct. The eight verified correct matches were determined by some form of confirmation such as through a street-based identity check. The study paid particular attention to the risks of such technology interfering with fundamental human rights, including privacy violations and discrimination due to algorithmic bias relating to sex, race, or ethnicity.

This study highlights the fact that any automated system can have some false positives and false negatives, which demands that human analysts are in the loop to manage the risks of something important being overlooked or misinterpreted. Good practice here can also help deal with the risk of contextual bias. An automated system supporting forensic analysis should offer sequential unmasking capabilities.

Ultimately, automated systems supporting forensic analysis should integrate forensic requirements, risk management, and privacy, i.e., forensic-by-design. If a system is forensic-by-design, forensic preparedness principles and practices have been integrated into the system engineering lifecycle, including risk management, incident handling, forensic principles, and legal compliance. However, such forensic-by-design is not common practice at the moment, and there are growing concerns about the unreliability and invasiveness of such technologies for criminal investigation purposes. Some cities in California are strictly controlling the acquisition and utilization of "any software, electronic device, system utilizing an electronic device, or similar device used, designed, or primarily intended to collect, retain, process, or share audio, electronic, visual, location, thermal, biometric, olfactory or similar information specifically associated with, or capable of being associated with, any individual or group."

Forensic laboratories must be prepared to address concerns about reliability, privacy, and discrimination associated with use of technology for processing data, particularly AI/ML systems for data analysis.

Digital reinforcement of forensic science principles

Core principles and processes in forensic laboratories can be bolstered using technology while increasing the efficiency and quality of services. These core principles include authenticity and integrity; reliability and reproducibility; and quality and efficiency. In addition, intelligent application of digital technology can create new opportunities to deal with crime in digitalized society.

Authenticity and integrity

Digital transformation of forensic laboratories can enhance the traceability of traces. Existing computerized systems for processing physical and digital evidence typically provide a digital audit trail and its treatment throughout its lifecycle in the forensic laboratory. Alterations to data files (raw and processed) might be detectable using the digital traces that are created routinely during the processing of exhibits in a forensic laboratory. In some situations, to authenticate data files and associated results, it is necessary to perform contextual analysis, including digital forensic analysis of the original data and metadata.

Contextual reconstruction scenario

In this scenario, questions have arisen about the results of some samples processed by a laboratory. In order to assess the reported results, the raw data was recovered from analysis systems along with processed results. The information from these files was compared with details recorded in the LIMS, and discrepancies were found. However, audit logs in the LIMS were found that corresponded with the original data. Timestamps of the original data files and the LIMS were also examined to determine contemporaneous activities versus later changes.

As this scenario highlights, it is advisable not to leave evidence authentication to chance. With properly protected audit trail and data file integrity mechanisms (preferably automated), a digitalized chain of custody can be maintained from evidence intake to evidence return or archiving, providing valuable insights into laboratory operations and helping protect against undetected mistakes and forensic fraud. Some forensic laboratories have adopted bar code scanning and RFID labels to mark and track exhibits throughout their lifecycle. Some forensic laboratories use secure storage to retain raw data files. Some laboratories are developing enhanced methods for maintaining provenance information using blockchain-based systems.

With such automated provenance tracking mechanisms in place, every deliverable that a forensic laboratory produces could include the associated electronic chain of custody details as an appendix to demonstrate proper handling. Providing this provenance information in a standardized format such as CASE enables receiving organizations to integrate the information into their information management systems and detect potential inconsistencies more easily, and even automatically.

Reliability and reproducibility

Forensic science practices demand that the data, methods, tools, and analysis are described in sufficient detail that they can be carried out again with the same results.

Digital transformations can support reproducibility of processing by documenting all phases of the evidence lifecycle in the forensic laboratory, as well as reproducibility of analysis by providing others with original data to perform independent analysis. Some software developers also provide a standalone application for viewing the results of forensic processes, enabling others to verify forensic analysis without requiring them to purchase licenses for the specific software and/or version.

Reproducibility of forensic analysis scenario

In this scenario, multiple parties need to perform an in-depth review of the results produced by a forensic laboratory, as part of an investigation involving various computers and mobile devices. To save cost and time, the forensic laboratory provided the multiple parties with a complete package of data and viewing software necessary to perform in-depth review. Using this approach, all parties could assess the reliability of the evidence and results without having to purchase specialized equipment and software, or to run time consuming data processing operations that had already been performed by the laboratory.

This scenarion shows that expanding decentralisation of forensic capabilities is driving the need for data and analysis results to be transferred securely between systems and organizations that is fully traceable, if not completely repeatable. This trend raises the importance of international standards for harmonization of forensic methods and data formats.

Quality and efficiency

Technological advances enable forensic laboratories to process evidence more quickly while maintaining quality. Forensic laboratories can manage the systems that they rely on for processing evidence in order to standardize the processing and output. The National Institute of Standards and Technology (NIST) Computer Forensic Tool Testing (CFTT) program publishes open access test reports for software commonly used in digital forensic laboratories, detailing both capabilities and limitations. The European Network of Forensic Science Institutes (ENFSI) and the US DoD Cyber Crime Center (DC3) also perform tool validation and testing, and provide the results to law enforcement agencies. Similar validation and testing should be applied to computers and software used to operate traditional forensic equipment. As with the prior discussed scenario of computer system malfunction, such validation must be alert for seemingly insignificant changes that can create major errors in traditional forensic processes. Furthermore, the validation of AI/ML based software is an important and complex undertaking that must be performed by an independent entity or consortium. This level of control helps improve the consistency and quality of results from forensic laboratory processes.

Validation and change control scenario

In this scenario, image that a forensic laboratory routinely validates and tests equipment, and maintains strict change control procedures for computers used to process evidence. Every aspect of the systems gets covered, including the specific hardware, firmware, and software versions. Validation of the forensic processes gets performed on the specific systems, and any issues are recognized and dealt with before causing further problems.

Forensic-by-design automation

Forensic laboratories are developing new automated systems, including those with AI/ML capabilities, to analyze data and to gain understanding of crime in ways that were previously not feasible.

Forensic artificial intelligence scenario

In this scenario, imagine that electronic portable devices, based on near-infrared spectroscopy and supported by machine learning, have been developed to test drugs rapidly. These devices could be useful for producing a full profile of the substance tested in the field and transfer the data to a computerized repository in the laboratory for further analysis. Such a repository could be useful for tracking drug trends in a region, and quickly detecting a problem such as an epidemic.

The value of forensic laboratories can be extended to more proactive intelligence-led approaches based on knowledge extracted from repetitions found in crime. To manage the associated risks, systems supporting such intelligence-based solutions should be forensic-by-design, abiding by forensic principles and human rights, including privacy and nondiscrimination. Forensic-by-design includes traceability, encryption, and impossibility even for system administrators to access private information.

Digital transformation risk management

This section describes a series of recommendations (Table 1) for managing risks of digital transformation of forensic laboratories. Forensic laboratories need to prepare for matters such as lost evidence, being audited or investigated, and responding to civil lawsuits. In the digital forensic domain, the practice of forensic preparedness has been developed to manage risks associated with computer use and misuse. The practice of forensic preparedness involves specification of a policy that lays down a consistent approach, detailed planning against typical (and actual) audit or investigative scenarios that an organization faces, identification of (internal or external) resources that can be deployed as part of those plans, identification of where and how the associated digital evidence can be gathered that will support investigation, and a process of continuous improvement that learns from experience.

Table 1. Road-map for digital transformation risk management Risk Recommendation Associated subsection (below) Not knowing what data exist Develop a digital evidence map documenting where needed data, logs, and provenance details are located and how they will be preserved in a forensic manner. "Curated digital information" Not having a plan Institute policies and procedures that govern the roles, responsibilities, and expected actions during an inquiry. "Established processes and procedures" Missing data and metadata Implement an automated digital forensic preservation process of primary data sources. "Data integrity" Lack of traceability Maintain and forensically preserve detailed electronic audit logs of all computer system usage and database transactions. "Strategic audit logging" Lack of data integrity Tamperproof the digitalized chain of custody ledger to support authentication of data files and audit logs. "Data authentication" Lack of practice Use audit logs and digitalized chain of custody records for routine purposes to ensure that the logs are monitored and used regularly. "Regular use or review of data and processes" Insufficient validation Validate computer-reliant forensic processes whenever there are changes to hardware or software configurations. "Test and validation" Black-box automation Evaluate automated systems at three levels: performance, understandability, and scientific interpretation. "Effective use of automation" Lack of digital forensic support Engage digital forensic expertise in preparation for problems, and to assist with dealing with problems that arise in a forensic laboratory. "Role of digital forensic expertise"

Notes

This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added.