View source for Journal:Development and national scale implementation of an open-source electronic laboratory information system (OpenELIS) in Côte d’Ivoire: Sustainability lessons from the first 13 years

{{Infobox journal article
|name         = 
|image        = 
|alt          = <!-- Alternative text for images -->
|caption      = 
|title_full   = Development and national scale implementation of an open-source electronic laboratory information system (OpenELIS) in Côte d’Ivoire: Sustainability lessons from the first 13 years
|journal      = ''International Journal of Medical Informatics''
|authors      = He, Yao; Liams-Hauser, Casey; Assoa, Paul H.; Kouabenan, Yves-Rolland; Komena, Pascal; Pongathie, Adama; Kouakou, Alain; Kirn, Mary; Antilla, Jennifer; Rogosin, Carli; Ngatchou, Patricia S.; Kohemun, Natacha; Koffi, Jean B.; Flowers, Jan; Abiola, Nadine; Adjé-Touré, Christiane; Puttkammer, Nancy; Perrone, Luca A.
|affiliations = University of Washington, I-TECH Côte d’Ivoire, Ministry of Health and Public Hygiene, U.S. Centers for Disease Control and Prevention
|contact      = lucy dot perrone at ubc dot ca
|editors      = 
|pub_year     = 2023
|vol_iss      = '''170'''
|at           = 104977
|doi          = [http://dx.doi.org/10.1016/j.ijmedinf.2022.104977 10.1016/j.ijmedinf.2022.104977]
|issn         = 1872-8243
|license      = [http://creativecommons.org/licenses/by-nc-nd/4.0/ Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International]
|website      = [https://www.sciencedirect.com/science/article/pii/S138650562200291X https://www.sciencedirect.com/science/article/pii/S138650562200291X]
|download     = [https://www.sciencedirect.com/science/article/pii/S138650562200291X/pdfft?md5=82e4616ec7b6ac38d1a42634d562efc6&pid=1-s2.0-S138650562200291X-main.pdf https://www.sciencedirect.com/science/article/pii/S138650562200291X/pdfft]
}}
{{ombox	 
| type      = notice	 
| image     = [[Image:Emblem-important-yellow.svg|40px]]	 
| style     = width: 500px;	 
| text      = This article should be considered a work in progress and incomplete. Consider this article incomplete until this notice is removed.	 
}}
==Abstract==
'''Purpose''': Côte d'Ivoire has a tiered [[public health laboratory]] system of nine [[Reference laboratory|reference laboratories]], 77 [[Laboratory|laboratories]] at regional and general [[hospital]]s, and 100 laboratories among 1,486 district health centers. Prior to 2009, nearly all of these laboratories used paper registers and reports to collect and report laboratory data to clinicians and national disease monitoring programs.

'''Project''': Since 2009 the Ministry of Health (MOH) in Côte d'Ivoire has sought to implement a comprehensive set of activities aimed at strengthening the laboratory system. One of these activities is the sustainable development, expansion, and technical support of an open-source electronic [[laboratory information system]] (LIS) called [[OpenELIS]], with the long-term goal of Ivorian technical support and managerial sustainment of the system. This project has addressed the need for a comprehensive, customizable, low- to no-cost, open-source LIS to serve the [[public health]] systems, with initial attention to HIV clients and later expansion to cover the general population. This descriptive case study presents the first published summary of original work which has been ongoing since 2009 in Côte d’Ivoire to transform the [[information management]] systems and processes in laboratories nationally.

'''Impact''': OpenELIS is now in use at 106 laboratories across Côte d’Ivoire. This article describes the iterative planning, design, and implementation process of OpenELIS in Côte d'Ivoire, and the evolving leadership, ownership, and capacity of the Ivorian MOH in sustaining the system. This original work synthesizes lessons learned from this 13-year experience towards strengthening LISs in other low-resource settings.

'''Highlights of this work''':<br />
* Factors for scaling and sustaining an electronic LIS (eLIS) in low- and middle-income countries (LMICs) have not been previously described in the literature.
* Successful adoption, scaling, and sustainment of OpenELIS in Côte d'Ivoire relied on early collaboration with partners, regulatory agencies, and technical experts into every step of the design, implementation, and evaluation phases.
* Entities planning to nationally scale an eLIS should plan for 1) workforce development in both end users and system administrators; 2) financial sustainability; and 3) institutionalization of government ownership and technical leadership.
* An open-source eLIS can increase accuracy and timeliness of clinical laboratory data, supporting diagnostic testing and monitoring in a resource-limited setting such as Cote d’Ivoire.

'''Keywords''': electronic information systems, laboratory information systems, quality, low-resource country

==Introduction==
In low- and middle-income countries (LMICs), inadequate infrastructure—including limited availability and use of electronic [[laboratory information system]]s (LISs)—has hindered quality [[laboratory]] service delivery for communities. [1] International development efforts such as the U.S. President's Emergency Fund for AIDS Relief (PEPFAR) have aimed to strengthen national laboratory systems by modernizing infrastructure and diagnostic testing methods, including automated analyzers that require computer connectivity for [[Information management|data management]]. The increase in the quantity and complexity of data generated from high-throughput analyzers and sophisticated diagnostics necessitates the transition from paper records to LIS.

Compared to paper-based systems, LIS are more efficient and enable better [[quality control]] (QC) in collecting, processing, managing, synthesizing, and reporting large amounts of data (Table 1). [1–2] The [[COVID-19]] [[pandemic]] has highlighted the acute importance of digital health, including LIS, in providing data and enabling rapid data exchange and sharing to facilitate [[public health]] surveillance and data-driven decision-making. [3]

{| 
 | style="vertical-align:top;" |
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="80%"
 |-
  | colspan="4" style="background-color:white; padding-left:10px; padding-right:10px;" |'''Table 1.''' Strengths of laboratory information systems (LISs) compared to paper-based processes.
 |-
  ! style="background-color:#dddddd; padding-left:10px; padding-right:10px;" |Lab aspect
  ! style="background-color:#dddddd; padding-left:10px; padding-right:10px;" |LIS feature(s)
  ! style="background-color:#dddddd; padding-left:10px; padding-right:10px;" |Actions
  ! style="background-color:#dddddd; padding-left:10px; padding-right:10px;" |Impact
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |'''Reception'''
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Batch entry; referral workflows
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Accelerate the processing and recording of incoming work
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Reduce turnaround time (TAT)
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="2"|'''Results generation'''
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Work plans
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Help managers monitor workload and supply management
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Ensure efficient operation of laboratories
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Analyzer interfaces
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Relay results quickly and accurately
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Reduce TAT
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="4"|'''Reporting'''
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Interface with [[electronic medical record]] (EMR)
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Communicate client laboratory history rapidly to clinicians in referral networks
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Enable faster clinical decision-making regarding diagnosis, treatment, and prognosis; reduce TAT
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Interface with client notification systems via Short Message Service (SMS) or email
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Communicate test results to clients rapidly
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Prompt immediate behavior augmentation and achieve better health outcomes (e.g., initiate self-quarantine in the case of COVID-19).
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Interface with routine health information systems at the district, regional, and national levels
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Enable faster aggregate data reporting
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Facilitate data-driven policy making; reduce TAT
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Reports of quality indicators such as turnaround time
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Help managers monitor quality performance
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Facilitate continuous quality improvement; increase testing quality
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="2"|'''Overarching'''
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Graphic widgets (e.g., drop-down menus, checkboxes), active logic and value checks, flags and reminders for potential issues, and automation
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Reduce errors
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Facilitate quality control; increase testing quality
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Data storage and data query tools
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Allow secure, long-term [[Data retention|data retainment]] and easy data retrieval based on specific needs
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Improve client privacy, long-term data availability, and data usage
 |-
|}
|}

However, LIS uptake and routine usage in LMICs remains low due to several barriers. LIS ownership, development, and maintenance can be perceived as bearing high costs of financial and human resources. Numerous proprietary LIS exist but may not be ideally suited for LMICs, where financial resources are limited and, most often, ongoing software support relies on long-term service contracts with private companies. Although some LMICs have implemented proprietary systems [4–6], these systems are vulnerable to market changes and can leave users unsupported if a company suspends its operations or modifies its service terms. Health information systems (HIS), including LIS, that use open-source code and are supported by communities of practice (CoPs) offer an attractive alternative in LMICs. Open-source CoPs have a vested interest in capacity building of software developers and users, including those from LMICs. [7] Implementation time and effort aside, there are no licensing fees, and all software code created is in public ownership, which enables others to customize and improve functionality. [8] Several open-source LISs are currently in use in LMICs in Africa and Asia, with varied scale. [9]

Another barrier to sustained, wide-scale usage of LIS is that LIS initiatives are often international-donor-driven and may lack close collaborations with ministries of health (MOHs) in LMICs. Engaging the MOH in the initial design and feature prioritization stage, building local capacity to own and maintain the software, and securing resource commitment after phaseout of donor support are time- and labor-intensive. [10] Additionally, power asymmetries between donors and LMIC stakeholders may discount local priorities, requirements, and innovations. [11] However, successful experiences of implementing an LIS in LMICs at the national scale have shown that strengthening MOH leadership and cultivating mutually beneficial partnerships are essential to impactful and sustainable laboratory system strengthening. [4][12]

Since 2009, the University of Washington’s (UW) International Training and Education Center for Health (I-TECH) has worked in partnership with the Côte d'Ivoire Ministry of Health and Public Hygiene (Ministère de la Santé et de l’Hygiène Publique, or MSHP) to implement a comprehensive set of activities aimed at strengthening the laboratory system. PEPFAR and the U.S. [[Centers for Disease Control and Prevention]] (CDC) have funded the partnership. One of the key activities is the sustainable development, expansion, and technical support of an open-source enterprise-level LIS, the [[OpenELIS]] system, with the long-term goal of Ivorian technical support and managerial sustainment of the system. The software serves as both an effective laboratory information management solution and a business operations framework for the laboratory service units (Fig. 1). The collaboration has addressed the need for a comprehensive, customizable, low- to no-cost, open-source LIS to serve the public health systems, with initial attention to HIV clients and later expansion to cover the general population.


[[File:Fig1 He IntJofMedInfo2023 170.jpg|1000px]]
{{clear}}
{| 
 | style="vertical-align:top;" |
{| border="0" cellpadding="5" cellspacing="0" width="1000px"
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |<blockquote>'''Figure 1.''' OpenELIS laboratory software solution and business process framework. (Color should be used in print).</blockquote>
 |- 
|}
|}

Côte d'Ivoire has a tiered [[public health laboratory]] system of nine [[Reference laboratory|reference laboratories]], 77 laboratories at regional and general [[hospital]]s, and 100 laboratories among 1,486 district health centers. Prior to 2009, most laboratories used paper registers and reports to collect and report laboratory data to clinicians and national disease monitoring programs. The only LIS was a bespoke database located at the national HIV reference laboratory (called rETRO-CI), and this system was paired with commercial software.

The objectives of this case study are to: 1) present the iterative implementation process of OpenELIS in Côte d'Ivoire; 2) describe the evolving leadership, ownership, and capacity of the Côte d'Ivoire MSHP in sustaining routine use of OpenELIS; and 3) synthesize lessons learned for strengthening LIS in other LMICs.

==Materials and methods==
We used qualitative methods in this case study to describe the implementation and collaboration around OpenELIS and its supporting activities. [12–13] We reviewed monthly and annual project reports from the implementers; activity and trip reports from technical experts, as well as from government and donor representatives; and software development roadmap and technical documentation from 2009 to 2020. This project was determined to be non-research by the University of Washington and has been approved by Côte d'Ivoire Comité National d'Ethique des Sciences de la Vie et de la Santé (CNESVS, Ivorian Institutional Review Board; reference number 006–21/MSHP/CNESVS-km) and the U.S. CDC.

Information abstracted from the project documents was initially summarized in chronological order and later coded deductively and inductively. Next, we coded the summaries deductively using the components of the Stages of Continuous Improvement (SOCI) Framework for Health Information Systems. [14] The SOCI Framework has previously been used in Uganda and Cameroon to assess the maturity of digital health tools.[14–16] In addition to deductive codes, we also created new codes inductively where the SOCI Framework did not apply. Inductive codes captured the different steps of OpenELIS implementation and evolving collaboration among stakeholders. When coding was complete, codes and their corresponding excerpts from the summaries were organized into themes. YH coded the summaries. The other co-authors reviewed and verified the coding and analysis to ensure accuracy. The coding and thematic analysis took place in ATLAS.ti 8 Windows.

To ensure that LIS implementation translates into high-quality health services and population health benefits [17], it is important to monitor how data systems mature over time. As such, the SOCI Framework outlines 13 components and 39 subcomponents within five domains, i.e., HIS leadership and governance, management and workforce, infrastructure, standards and interoperability, and [[data quality]] and use.[18–19]

==Results==
===Design and development===
OpenELIS was initially designed and implemented at two U.S. state public health laboratories in Minnesota and Iowa. [20] Its implementation in global health began as part of a collaborative effort between the U.S. CDC and the Government of Vietnam to strengthen HIV service delivery and laboratory systems in 2005. [12] As part of the open-source medical record system (OpenMRS) consortium of partners [21], UW I-TECH received the OpenELIS source code and joined the development of the OpenELIS codebase in 2009.

An agile software design and development methodology was adopted by UW I-TECH for OpenELIS to enable an inclusive, iterative approach to software creation together with the local stakeholders in Côte d'Ivoire.[22] Since its introduction and adaptation in Côte d'Ivoire in 2009, OpenELIS has evolved through multiple iterations of improving functionality and flexibility, and through new releases twice per year. The first versions of OpenELIS were limited in scope and focused primarily on HIV care. They provided pre-defined forms and reports for common HIV-related tests such as early infant diagnosis of HIV (EID), CD4 counts, and HIV genotyping.

In 2011, UW I-TECH merged all versions of OpenELIS for different laboratories into one core, i.e., OpenELIS Global 1.4, greatly reducing the burden of maintenance and update. Before 2011, we created a customized copy of OpenELIS for each laboratory. This approach was manageable when OpenELIS was implemented in four national laboratories in Côte d'Ivoire and Haiti. However, as OpenELIS expanded to new laboratories, the time burden of maintaining and updating the software across different versions became excessive. The shift to a common code base shortened the software testing cycle for each new version from approximately 200 hours to 40 hours and permitted a more thorough software testing process and a higher-quality final release.

The core software improvements also enabled greater customizability and made it possible for a lab to install and configure OpenELIS without engaging a software developer to customize elements. We added test catalog management features, allowing laboratories to modify catalogs between version releases. Laboratories could make changes to existing tests or add new tests or panels without a software developer adding them manually to the system. New analyzer interfaces were added so that laboratories could import data from analyzers beyond those related to HIV. The core software could be deployed without impacting previous local customizations and further adapted to any setting without code re-write or maintenance. This allowed for countries and laboratories to have more control over what features to include in the software.

In 2018, UW I-TECH began an extensive review of the OpenELIS application and identified where the older technology had become obsolete and presented data security risks. OpenELIS Global 2.0, completed in 2019, upgraded the core framework to mitigate security risks (Table 2).

{| 
 | style="vertical-align:top;" |
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="80%"
 |-
  | colspan="3" style="background-color:white; padding-left:10px; padding-right:10px;" |'''Table 2.''' Open-source components in OpenELIS. HL7 = [[Health Level 7]]; FHIR = [[Fast Healthcare Interoperability Resources]]; ASTM = American Society of Testing and Materials
 |-
  ! style="background-color:#dddddd; padding-left:10px; padding-right:10px;" |Component
  ! style="background-color:#dddddd; padding-left:10px; padding-right:10px;" |OpenELIS Global 1.X series
  ! style="background-color:#dddddd; padding-left:10px; padding-right:10px;" |OpenELIS Global 2.X series
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |'''Timeframe'''
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Before 2011
  | style="background-color:white; padding-left:10px; padding-right:10px;" |After 2011
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |'''Operating system'''
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Ubuntu 12
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Ubuntu 20
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |'''Programming framework'''
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Java Struts 1
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Java Spring
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |'''Web application'''
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Tomcat
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Tomcat
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |'''Database'''
  | style="background-color:white; padding-left:10px; padding-right:10px;" |[[PostgreSQL]]
  | style="background-color:white; padding-left:10px; padding-right:10px;" |PostgreSQL
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |'''Interoperability'''
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Mirth, HL7 2.5.1
  | style="background-color:white; padding-left:10px; padding-right:10px;" |FHIR R4, HL7 2.5.1, ASTM
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |'''Customization'''
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Hard-coded multiple forks by developers for each use case
  | style="background-color:white; padding-left:10px; padding-right:10px;" |Core software with flexible customization by users
 |-
|}
|}

Many expansions and improvements in later iterations of OpenELIS stemmed from close collaboration between UW I-TECH and the multi-stakeholder OpenELIS technical working group (TWG) in Côte d'Ivoire, convened in 2015. The TWG consists of representatives from the MSHP, I-TECH Côte d'Ivoire, PEPFAR's NGO implementing partners (IPs), and the CDC. UW I-TECH and the design team within the TWG would lead design workshops that start with needs gathering at regional and district hospitals and laboratories using a standardized questionnaire. After reconvening, group members would share the findings on identified needs and gaps in LIS functionality and reporting, prioritizing design features, adding the designs to the roadmap [23], and writing software specifications.

The software development process followed a similar collaborative trajectory as the design process. Before 2014, UW I-TECH conducted all software development, mostly by one senior software developer and joined later by a second developer. All reports, enhancements, and changes relied on the two developers and was dependent on the next release. Starting in 2015, UW I-TECH has provided software development training in Côte d'Ivoire and created a more collaborative workflow with Ivorian developers so that both design and development have become more localized.

===Deploy, scale-up, and support===
OpenELIS deployment and scale-up took place in several stages since 2009 (Table 3), and the respective roles of the MSHP and I-TECH in the process evolved over time. Before 2015, I-TECH led the deployment process. The deployment at a laboratory began with a team of I-TECH staff from Côte d'Ivoire and Seattle conducting an initial laboratory assessment. Then, during a second visit, the team deployed a customized version. By the end of 2015, OpenELIS had been deployed at 11 laboratories, covering the entire network of laboratories performing viral load testing at the time (Fig. 2).

{| 
 | style="vertical-align:top;" |
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="100%"
 |-
  | colspan="3" style="background-color:white; padding-left:10px; padding-right:10px;" |'''Table 3.''' Timeline of the OpenELIS scale-up in Côte d'Ivoire.
 |-
  ! style="background-color:#dddddd; padding-left:10px; padding-right:10px;" rowspan="2"|Year
  ! style="background-color:#dddddd; padding-left:10px; padding-right:10px;" colspan="2"|Number of laboratories implementing OpenELIS
 |-
  ! style="background-color:#dddddd; padding-left:10px; padding-right:10px;" |New
  ! style="background-color:#dddddd; padding-left:10px; padding-right:10px;" |Cumulative
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |2009–2012
  | style="background-color:white; padding-left:10px; padding-right:10px;" |3
  | style="background-color:white; padding-left:10px; padding-right:10px;" |3
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |2013–2015
  | style="background-color:white; padding-left:10px; padding-right:10px;" |8
  | style="background-color:white; padding-left:10px; padding-right:10px;" |11
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |2016
  | style="background-color:white; padding-left:10px; padding-right:10px;" |1
  | style="background-color:white; padding-left:10px; padding-right:10px;" |12
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |2017
  | style="background-color:white; padding-left:10px; padding-right:10px;" |31
  | style="background-color:white; padding-left:10px; padding-right:10px;" |43
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |2018
  | style="background-color:white; padding-left:10px; padding-right:10px;" |12
  | style="background-color:white; padding-left:10px; padding-right:10px;" |55
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |2019
  | style="background-color:white; padding-left:10px; padding-right:10px;" |13
  | style="background-color:white; padding-left:10px; padding-right:10px;" |68
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |2020
  | style="background-color:white; padding-left:10px; padding-right:10px;" |28
  | style="background-color:white; padding-left:10px; padding-right:10px;" |93
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |2021
  | style="background-color:white; padding-left:10px; padding-right:10px;" |15
  | style="background-color:white; padding-left:10px; padding-right:10px;" |111
 |-
|}
|}


[[File:Fig2 He IntJofMedInfo2023 170.jpg|1000px]]
{{clear}}
{| 
 | style="vertical-align:top;" |
{| border="0" cellpadding="5" cellspacing="0" width="1000px"
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" |<blockquote>'''Figure 2.''' Laboratories in Côte d'Ivoire that implement OpenELIS before and after 2015. (Color should be used in print).</blockquote>
 |- 
|}
|}

In 2015, after the MSHP defined and ratified LIS requirements, they determined that OpenELIS satisfied most of these requirements and would be the system deployed at laboratories throughout the country. The MSHP became the owner of OpenELIS in Côte d'Ivoire and started collaborating with I-TECH closely in deployment to roll out OpenELIS. The scale-up first prioritized regional referral laboratories that newly started providing viral load testing as part of a MSHP- and CDC-led initiative to scale-up viral load testing. OpenELIS was then deployed in hospitals at the district and local level to support their collection and management of data on routine laboratory testing for both HIV and non-HIV clients. As of 2021, 111 laboratories in Côte d'Ivoire had implemented OpenELIS (Fig. 2). OpenELIS currently supports testing of [[hematology]], [[biochemistry]], [[molecular biology]], serology, and testing of [[microbiology]], [[pathology]], and immunohistochemistry will be available in January 2023.

The collaborative deployment and scale-up process started with a series of deployment workshops with relevant stakeholders from the OpenELIS TWG in 2015. Participants received hands-on training on setting up servers and workstations, conducting maintenance and updates, and troubleshooting. During the deployment wave each year, starting from 2015, a team of MSHP implementation, I-TECH Côte d'Ivoire staff, and IP representatives would organize a week-long implementation trip to deploy the software and perform the initial training of laboratory staff. The team would follow up with the laboratories by phone weekly and conduct a coaching and review visit pne to three months after the deployment and training.

Starting in 2020, the MSHP began leading the deployment process with minimal support from I-TECH. The MSHP team conducts all follow-up visits, and the training is facilitated by MSHP trainers with co-facilitation by I-TECH Côte d’Ivoire. The MSHP also independently operates the OpenELIS Technical Assistance Unit (Cellule d'Assistance Technique de OpenELIS, or CATOE), the technical support call center for OpenELIS. Established in 2016 within the MSHP, CATOE provides direct, real-time technical support to OpenELIS users. CATOE technicians hold bidirectional call services with laboratories to conduct monthly check-ins that proactively probe for issues and to respond to technical assistance requests. CATOE consists of four MSHP staff members who were recruited from the participants of the first OpenELIS implementers' workshop in 2015. The technicians received targeted training on providing real-time support, equipment, phone credits, and a six-month mentorship with the primary support and development staff at I-TECH. CATOE meets monthly with the OpenELIS TWG to review activities and findings from the support calls, feeding information into the software roadmap.

===Strengthening local leadership and ownership===
The Côte d'Ivoire MSHP started collaborating with I-TECH in OpenELIS implementation more actively in 2015. The MSHP initiated a reorganization process in 2014 that spurred the creation of the Department of Informatics and Health Information (Direction de l'Informatique et de l'Information Sanitaire, or DIIS) in 2015 to consolidate and coordinate health informatics activities in the country. This designation signaled a leadership commitment to ownership of HIS development, implementation, and management activities nationally. In 2017 the MSHP incorporated the designation of OpenELIS and its scale-up in the National Strategic Plan for the Development of Medical Biology Laboratories 2017–2020. [24]

The continuous strengthening of MSHP leadership demonstrates comprehensive ownership of not only the software and hardware but also the essential knowledge, skills, and processes to sustain routine OpenELIS implementation. Before 2015, the MSHP was only involved in site selection for OpenELIS. After the formation of DIIS, the MSHP joined I-TECH throughout the implementation process from forming the TWG, gathering needs, and training developers and technicians to deploying the software and providing technical support. By 2020, the DIIS was conducting follow-up visits and operating CATOE independently, leading the deployment process, configuration of servers, and training of new and existing users.

===Building local capacity===
In addition to providing the software and hardware and strengthening local leadership, building local capacity for implementing OpenELIS in Côte d'Ivoire is also critical. Training on various topics along the software development pipeline and in various formats has equipped Ivorian policymakers and technicians with necessary knowledge and skills for routine OpenELIS implementation and maintenance.

Before 2015, UW I-TECH staff initially provided on-the-job training (OJT) on OpenELIS deployment and upgrade to selected staff at the MSHP, national laboratories, and IPs. In 2014, a targeted month-long training on OpenELIS development took place in Seattle for software developers with background in Java from the MSHP, Institut Pasteur Côte d'Ivoire, Institut National Polytechnique Félix Houphouët-Boigny (INPHB; a public polytechnic institute of higher education), and I-TECH Côte d'Ivoire. After participants returned to Côte d'Ivoire, UW I-TECH staff provided remote mentoring and exercises. While the results of the OJT and targeted training were promising, the time and resources for replicating these formats became unsustainably high as the OpenELIS scale-up started in 2015.

In response, UW I-TECH started collaborating with the MSHP in 2015 to develop a full suite of materials for OpenELIS, including a package for trainers and users, and to lead a series of training of trainers (TOT) sessions within Côte d'Ivoire. The week-long TOTs used theoretical and practical activities and took a holistic technical approach covering a range of topics from design, planning, development, and programming to implementation and support. Among the participants at the first TOT, one became the deputy director of DIIS, and most joined the OpenELIS TWG. The TOT participants later conducted cascade training led by the MSHP during the OpenELIS scale-up. Cascade training participants provided largely positive ratings, appreciating the theoretical and practical exercises. The TOTs and cascade trainings have been an integral part of the deployment and scale-up process since 2015, and they have formed a cadre of trained teams in Côte d'Ivoire to ensure sustainable OpenELIS deployment and support.

The collaboration between I-TECH and INPHB incorporated OpenELIS into the computer science curriculum and started the OpenELIS internship at INPHB. In 2016, two INPHB faculty received training on OpenELIS software development from UW I-TECH, and they started using OpenELIS as an example LIS to teach students in computer science. In 2018, the OpenELIS internship program started with three students studying at INPHB and working with I-TECH Côte d'Ivoire simultaneously to practice OpenELIS software development, deployment, and technical support. The training program and assessment process became standardized in 2019, and two interns per year embedded with I-TECH or MSHP teams for one year. Ten interns have graduated from the program, and one was eventually hired as I-TECH Côte d'Ivoire staff.

In addition to periodic training and the internship program, software developer retreats and opportunities to attend and present at international and regional conferences for HIV/AIDS or [[health informatics]] have also been part of capacity building.

===Supporting data-driven decision-making in HIV/AIDS===
To transform data stored in OpenELIS into [[information]] that is meaningful for decision-making, I-TECH built the online data dashboards that display aggregate data on viral load testing and EID imported from OpenELIS. [25] The dashboards import data from OpenELIS and create visualizations that help the MSHP, IPs, and CDC monitor performance along the HIV care cascade and make programmatic decisions to improve quality of care. For example, the viral load dashboard helped discover a large group of clients who only had initial HIV diagnostic and viral load tests but no follow-up tests across multiple regions in Côte d'Ivoire. This issue would have been hidden before the dashboarding process cleaned, consolidated, and visualized the data. The MSHP and CDC based their directions for the regions and IPs on the discovery from the dashboard.

I-TECH also developed the pediatric HIV case management tool that combined OpenELIS data of pediatric HIV clients with those from the HIV [[electronic medical record]]s (EMR) system in Côte d'Ivoire. This tool helped health facilities and districts identify and follow up with lost-to-follow-up pediatric clients and parents and those whose viral load test results indicated a decline in antiretroviral regimen effectiveness. OpenELIS's pivotal role as an essential public health tool was solidified.

===Networking, interoperability between systems, and expansion===
To respond to emerging needs from the MSHP and laboratories, the OpenELIS project will always have room for adaptation and growth. Current activities include transitioning from a series of individual instances of OpenELIS to a more interconnected national laboratory information system through system networking. The pilot of a consolidated server for the national laboratory data warehouse began in 2021, bringing all laboratory data to one central location for reporting and real-time decision-making by the MSHP. The continuous demands from laboratories to add more testing modules to OpenELIS motivated the project to start developing modules for tuberculosis, COVID-19, and enhanced support for molecular biology and bidirectional analyzer interfacing. A reference testing workflow has been developed, allowing results to be sent between OpenELIS instances and providing laboratory results instantly between laboratories and reference laboratories to aid diagnosis and allow for faster treatment. Programmatically, I-TECH will continue to work with the Côte d'Ivoire MSHP to provide necessary assistance for the full transition.

==Discussion==
To our knowledge, this is the first case study of best practices and lessons learned from the implementation of a LIS in a tiered laboratory system in West Africa. This case study illustrates the development, implementation, and capacity building process for OpenELIS in Côte d'Ivoire since 2009, with support from PEPFAR, as part of an initiative to strengthen the national laboratory system. The open-source nature of OpenELIS coupled with the close partnership between the MSHP and I-TECH have facilitated the collaborative, iterative process of designing, developing, deploying, and scaling up OpenELIS in Côte d'Ivoire. Strong local leadership and ownership of OpenELIS implementation contributed to the scaling and sustainment of routine use of OpenELIS at laboratories nationwide beyond the first 13 years. The improved capacity and talent pool for OpenELIS implementation and support, resulting from continuous training efforts, also facilitated scaling and sustainment. OpenELIS and its dependent data dashboards have already demonstrated value in supporting the MSHP and other stakeholders in data-driven decision-making.

===Open-source design and CoPs-supported software adoption, implementation, sustainability, and innovation===
The open-source nature and the resulting potential for in-country capacity building and ownership of OpenELIS were the main reasons why the Côte d'Ivoire MSHP decided to adopt it. Reports from open-source LIS software projects in other settings, including the OpenELIS project in Vietnam [12], a LIS pilot for tuberculosis in Peru [26], and a LIS pilot for HIV in Malawi [27], have also underscored the value of open-source software in LMICs. Open-source software provide flexibility and agility so that governments can start with a pilot project and gradually build in more functions as use cases evolve and expand. The open-source nature invites collaboration and cultivates local capacity so that costs of development and maintenance will decrease over time, contributing to sustainability. Digital Square, a global consortium for promoting health equity through digital health, designated OpenELIS in 2018 as a digital “global good.” [28] A CoP was also developed to support the advancement and implementation of OpenELIS globally. As of October 2020, over 270 laboratories in 18 LMICs in sub-Saharan Africa, Asia, and the Caribbean use OpenELIS.

Continuous maintenance and funding support for software is crucial after the initial launch [29], and the OpenELIS CoP worldwide is a model that facilitates sustainable implementation, meaningful collaboration, and local ownership. There are three common models for implementing open-source software in LMICs, namely [[software as a service]] (SaaS) [30], donor investment in the core global good [31–32],  and CoP. [33] The first two models keep the knowledge and maintenance responsibility with the developers who are usually in high-income countries, and funding can be highly donor-driven. On the contrary, the CoP model brings a variety of actors and resources to a shared table to support the core stewardship and maintenance of the global good. The large-scale adoption of OpenELIS through different funding streams, not just PEPFAR, has led to a broader set of features and more sustainable cost-sharing, including recent co-investment from LMICs and the U.S. in 2021. Each implementing country benefits from features developed for other use cases due to the single code base and flexible administrative options. The OpenELIS CoP continuously supports the development of innovations and flexible design that can be used across use cases. [7]

===Collaboration, capacity building, and local ownership are the key to sustainable implementation===
Numerous global health projects have struggled to continue beyond the pilot or trial phase, and one of the key barriers to scale-up and sustainability is lack of local ownership and leadership at each level of the system. [34–35] The collaboration between the MSHP and I-TECH is the nurturing ground for the decade-long OpenELIS project in Côte d'Ivoire. The open-source and global CoP of software such as OpenELIS naturally support technical collaboration and exchange. [36] However, without committed engagement and close partnership with the MSHP, the OpenELIS design and development process in Côte d'Ivoire would likely have been less participatory, less successful, and less sustainable. [37]

The improved local capacity of software development and maintenance for OpenELIS is an important contribution toward the vision of long-term, locally sustained implementation. Providing information technology (IT) without investing in human resources or building capacity would still be inadequate in addressing the gaps in the laboratory system. [2] Therefore, the policymakers and technicians trained in OpenELIS development and deployment, as well as the interns embedded in MSHP teams to support OpenELIS implementation, will ensure the medium- and long-term success of the project.

The collaboration has also strengthened local ownership of OpenELIS and leadership of relevant software development and support activities to achieve routine usage and nationwide scale. Although the OpenELIS project in Côte d'Ivoire is still receiving funding by the U.S. government as part of PEPFAR, and the Côte d'Ivoire MSHP has not committed to completely taking over the continuous implementation of OpenELIS, there has at least been progress towards this goal. Transition of all OpenELIS-related planning, design, development, and implementation to the MSHP has been underway as of March 2022.

===Limitations===
This case study represents the experiences of one LIS project in Côte d'Ivoire and thus may not be generalizable to other settings, though the lessons learned are relevant to many HIS strengthening projects in LMICs. Evaluation research that uses rigorous quantitative, qualitative, or mixed methods is necessary to assess and demonstrate the reach, effectiveness, adoption, implementation, and maintenance of the project. [38] For example, one could evaluate the use of a quasi-experimental design to assess the effectiveness of improving laboratory testing data quality (e.g., data timeliness, data completeness, data validity) and service quality (e.g., TAT, testing validity) by comparing LIS to paper records, or the use of a qualitative study to interview laboratory staff about the facilitators and barriers to LIS implementation.

==Conclusions==
The development and nationwide scale-up of OpenELIS in Côte d’Ivoire for over a decade have relied on the open-source software design, CoP support, close collaboration with the Côte d’Ivoire MSHP, and cultivation of local ownership and capacity since inception. Other countries and supporting partners planning to adapt and nationally scale LIS may consider the importance of HIS workforce development, financial sustainability, and institutionalization of government ownership and technical leadership.

Before this study, the following was already known:
* LIS has the potential to increase accuracy and timeliness of laboratory data, supporting diagnostic testing and clinical monitoring.
* Factors for scaling and sustaining LIS in LMICs have not been previously described.

This study added the following to our knowledge:
* OpenELIS, an open-source electronic LIS, has been implemented in Côte d'Ivoire since 2009 and scaled nationwide to 111 clinical and reference laboratories.
* Successful adoption, scaling, and sustainment of OpenELIS has relied on active collaboration starting in the earliest planning phases and integrating local partners, regulatory agencies, and technical experts into every step of the design, implementation, and evaluation phases.
* Other countries and supporting partners planning to adapt and nationally scale LIS may consider the importance of carefully developing plans for 1) workforce development in both end users and system administrators; 2) financial sustainability; and 3) institutionalization of government ownership and technical leadership.

==Abbreviations, acronyms, and initialisms==

* '''ASTM''': American Society of Testing and Materials
* '''CATOE''': Cellule d'Assistance Technique de OpenELIS
* '''CDC''': Centers for Disease Control and Prevention
* '''CNESVS''': Côte d'Ivoire Comité National d'Ethique des Sciences de la Vie et de la Santé
* '''CoP''': community of practice
* '''DIIS''': Direction de l'Informatique et de l'Information Sanitaire
* '''eLIS''': electronic laboratory information system
* '''EID''': early infant diagnosis
* '''EMR''': electronic medical record
* '''FHIR''': Fast Healthcare Interoperability Resources
* '''HIS''': health information system
* '''HL7''': Health Level 7
* '''I-TECH''': International Training and Education Center for Health
* '''INPHB''': Institut Pasteur Côte d'Ivoire, Institut National Polytechnique Félix Houphouët-Boigny
* '''IP''': implementing partner
* '''LIS''': laboratory information system
* '''LMIC''': low- and middle-income country
* '''MOH''': Ministry of Health
* '''MSHP''': Ministère de la Santé et de l’Hygiène Publique
* '''OJT''': on-the-job training
* '''OpenELIS''': Open-source Electronic Laboratory Information System
* '''PEPFAR''': U.S. President's Emergency Fund for AIDS Relief
* '''SMS''': Short Message Service
* '''SOCI''': Stages of Continuous Improvement
* '''TAT''': turnaround time
* '''TOT''': training of trainers
* '''TWG''': technical working group
* '''UW''': University of Washington

==Acknowledgements==
We thank the Côte d'Ivoire MSHP and all implementing laboratories for the continuous collaboration. We thank Oliver Defawe, Paul Schwartz, Justin Sogo, Emily DeRiel for their programming and project planning support early in this work. We thank the CDC and our fellow PEPFAR IPs (ACONDA, Ariel Glaser Foundation, Elizabeth Glaser Pediatric AIDS Foundation, ICAP, SEV-CI, and Health Alliance International) for their collaboration in the implementation and optimization of OpenELIS in Côte d'Ivoire.

===Funding===
Funding for this work was provided by the U.S. Health Services and Resources Administration (#6U91HA06801) and CDC (#NU2GGH001968-05-00) awarded to the University of Washington.

===Conflict of interest===
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

==References==
{{Reflist|colwidth=30em}}

==Notes==
This presentation is faithful to the original, with only a few minor changes to presentation. Grammar was cleaned up for smoother reading. In some cases important information was missing from the references, and that information was added. No other changes were made in accordance with the "NoDerivatives" portion of the content license.

<!--Place all category tags here-->
[[Category:LIMSwiki journal articles (added in 2023)]]
[[Category:LIMSwiki journal articles (all)]]
[[Category:LIMSwiki journal articles on laboratory informatics]]