Journal:A benchmarking analysis of open-source business intelligence tools in healthcare environments

From LIMSWiki
Revision as of 23:43, 1 November 2016 by Shawndouglas (talk | contribs) (Saving and adding more.)
Jump to navigationJump to search
Full article title A benchmarking analysis of open-source business intelligence tools in healthcare environments
Journal Information
Author(s) Brandão, Andreia; Pereira, Eliana; Esteves, Marisa; Portela, Filipe; Santos, Manuel F.; Abelha, António; Machado, José
Author affiliation(s) University of Minho, Instituto Politécnico do Porto
Primary contact Tel.: +351-967-562-540
Editors Susilo, Willy
Year published 2016
Volume and issue 7 (4)
Page(s) 57
DOI 10.3390/info7040057
ISSN 2078-2489
Distribution license Creative Commons Attribution 4.0 International
Website http://www.mdpi.com/2078-2489/7/4/57/htm
Download http://www.mdpi.com/2078-2489/7/4/57/pdf (PDF)
This article should not be considered complete until this message box has been removed. This is a work in progress.

Abstract

In recent years, a wide range of business intelligence (BI) technologies have been applied to different areas in order to support the decision-making process. BI enables the extraction of knowledge from the data stored. The healthcare industry is no exception, and so BI applications have been under investigation across multiple units of different institutions. Thus, in this article, we intend to analyze some open-source/free BI tools on the market and their applicability in the clinical sphere, taking into consideration the general characteristics of the clinical environment. For this purpose, six BI tools were selected, analyzed, and tested in a practical environment. Then, a comparison metric and a ranking were defined for the tested applications in order to choose the one that best applies to the extraction of useful knowledge and clinical data in a healthcare environment. Finally, a pervasive BI platform was developed using a real case in order to prove the tool's viability.

Keywords: business intelligence; open-source; healthcare; benchmarking

Introduction

Increasingly, the healthcare sector is looking for computer applications to support the daily practice of health professionals. is particularly desirable in this sector as, besides being free, it has a source code that is fully available to users for viewing, reading, modification, and redistribution, without the restrictions of ownership of the product (unlike free software, which only allows its use without charge).

Open-source software differs fundamentally from an ownership model in terms of the development process and the product licenses. All open-source applications are licensed by an open-source license, which gives the user the right to use the software, access and modify the source-code, and redistribute the software for free. This type of software is very popular due to its many advantages, since it promises to accelerate the diffusion of information technology (IT) solutions in healthcare. Thereby, it can contribute to reduced development costs.[1][2]

One the other hand, the choice of a free license software in health and medical informatics is important because it determines the user’s rights and can influence the developers’ willingness to participate in a project, the quality of the product, and the willingness of users to adopt certain applications.[1][2]

In terms of costs, organizations can save on licensing fees and reduce expenditures on specific computer hardware. However, organizations need to welcome and train specialized collaborators in the adoption of open-source solutions. This type of situation has the hidden costs of highly skilled employees, implementation, maintenance, and a support process, which may lead to adopting proprietary solutions.[1][2]

The acquisition of a business intelligence (BI) tool and its implementation are quite advantageous for the health organization. Extending their use can have a positive global impact. However, the healthcare environment has some particularities that a BI solution should be prepared to answer. For example, the BI system could lead to resource optimization in various departments; it will improve the clinical condition of the patient through efficient diagnosis and the identification and application of the best practice protocols for treatment, among others.[3]

In order to help decision-makers make the best choices, a benchmarking study of BI tools focused on the healthcare environment was performed. After a thorough review of the literature, it was determined which tools were being used in this study. These tools were selected based on their good performance in several areas, such as management, healthcare, or retail.[4][5][6] Thus, the tools selected for this study were QlikView[7], Palo BI Suite[8], Jaspersoft BI[9], Tableau Public[10], SpagoBI[11], and Pentaho BI Suite.[12]

The analysis of this type of software emerged during a project that included the development of a BI platform in maternity care to visualize clinician and management indicators, as well as integrating data mining (DM) predictive models. All the tools were tested using real data provided by the Centro Hospital do Porto (CHP) and Centro Materno Infantil do Norte (CMIN). With this study, it is intended to select the BI tool that best suits the healthcare sector by using a practical case.

Besides the introduction, this article includes eight sections. The second section provides an approach to the background and related work, in which the concept of business intelligence is addressed and an introduction to the case study is presented. The third section is concerned with the application of BI tools in healthcare environments, followed by the requirements considered essential in these tools, in the section "Tool Requirements in Healthcare Environments." Thereafter, the next section addresses in more detail the tools selected; and the section "Business Intelligence Tools — Assessment Process" discusses our approach to the case study. Finally, the last sections correspond to the results, discussion, and conclusion.

Background and related work

Business intelligence

Business intelligence (BI) is the transformation of information stored in knowledge, enabling us to provide adequate information to a particular user at the appropriate time in order to support the decision-making process in real time. Thus, BI integrates a set of tools and technologies that enables the collection, integration, analysis, and visualization of data. For the implementation of a BI platform, it is necessary to perform some intermediate steps that are common in the development of this type of software tool, such as the construction of a data warehouse (DW). In this case, the Kimball methodology was chosen in order to design as well as develop and deploy the DW.[13] Thus, some of these steps are considered crucial for the successful implementation of a BI system, such as: tasks planning and expected results; defining the architecture that aims to follow the BI system; the selection and installation of the most appropriate BI tool; building the data warehouse dimensional model; the extraction, transformation and loading (ETL) process; and, finally, the development of the BI application.[13]

Thus, in order to develop a BI application, initially, it is necessary to choose the software that is most appropriate for achieving the desired outcomes. Thereby, it is necessary to undertake an analysis of most of the software available and choose the one type that provides the necessary and desired resources.[3][14][15]

Examples of application of BI tools in healthcare environments

With the urgency of acquiring medical informatics applications, open-source software is receiving increased attention from the healthcare industry. For example, the open-source project Care2X — composed of a hospital information system, practice management, a central data server, and a health exchange protocol — is under development in Europe. Care2X was developed in order to overcome integration problems in multiple incompatible network programs. It is possible to integrate almost any type of service, system, department, process, data, or communication of a hospital. Care2X supports the clinical workflow, incorporating diagnosis-related groups (DRG), as well as scheduling and electronic prescribing modules.[16]

openEHR corresponds to another case, which is sponsored by the openEHR Foundation. It promotes the "development of an open, interoperable health computing platform, of which the major component is clinically effective and interoperable electronic health records (EHRs)."[17]

Canada Health Infoway, established by Canadian federal and provincial grants, began an open-source initiative in 2005 to develop software that hospitals and developers could use to ensure the secure exchange of medical records of patients between various entities.[6] Thereby, these initiatives suggest that open-source is a viable way of developing applications in healthcare.[18][19]

In addition, it is noteworthy that currently there are already a few applications developed based on open-source tools, implemented in healthcare organizations, such as:

  • Turin ASL 3, which is a system developed in conjunction with the open-source Spago BI that allows the assignment of permissions for use according to the different types of users. It provides analytical documents, enables the data visualization, and also allows the use of the online analytical processing (OLAP) technology. This solution is implemented in local healthcare institutions and the Italian National Health Service. Turin ASL 3 was born in 1995 and is implemented in two hospitals in the city of Turin (Amadeo di Savoia and Maria Vittoria), Italy.[20]
  • St Antonius is an application developed by Pentaho BI Suite, built in order to analyze the waiting times of patients. The main advantages of this application are the improvement of operational efficiency, elimination of costs resulting from the creation of manual reporting, behavioral analysis, and the identification of patterns and risk analysis. This application has been implemented in the St. Antonius Hospital, located in Nieuwegein, Netherlands.[21]

Case study - Context

The analysis of business intelligence open-source software was one of the steps of a process conducted in order to develop a BI platform to support decision-making in maternity care in Centro Materno Infantil do Norte (CMIN). CMIN is one of the constituents of the Centro Hospitalar do Porto (CHP), along with the Hospital Santo António (HSA) and Hospital Joaquim Urbano (HJU).

The BI platform is directed at the modules of Gynecology and Obstetrics (GO) and Voluntary Interruption of Pregnancy (VIP) since these modules are lacking decision support systems. This platform aims to visualize the knowledge extracted from the data stored in information systems in CMIN, through their representation in tables, charts, and tables, among others, but also by integrating DM predictive models.

Some of the VIP key performance indicators (KPIs) that health professionals have interest in include:

  • characterization of the patient group by number of pregnancies and by date;
  • characterization of the patient group by number of children and by date;
  • characterization of the patient group by number of previous VIP experiences and by date;
  • characterization of the patient group concerning the revision consultation for date; and
  • characterization of the patient group based on contraception early in the process by date.

All the information recorded in the CHP is stored in different systems, taking as an example the nursing support system (SAPE), where a portion of the data recorded in the VIP module is stored, and the electronic health record (EHR), where all the patient data are stored.

The Medical Information Integration, Dissemination and Storing Agency (AIDA) guarantees the interoperability of these systems. AIDA is a system of intelligent agents that allows communication between different information systems in CHP.[22][23]

Applications of open-source BI tools in healthcare

With the computerization of clinical processes in healthcare organizations, the storage of clinical data in databases has been increasing exponentially. However, a lack of technology to gather, analyze, and distribute the most relevant information makes these organizations rich in data but extremely poor in information.[3] Nonetheless, forward-thinking healthcare organizations are aware that the data and its treatment through the business intelligence (BI) technology are essential for an informed and accurate decision-making process, as well as necessary to improve services and ensure the future of these organizations.

Cases of adoption of open-source BI tools by healthcare organizations are scarce. However, the great benefits arising from its implementation in other areas have led to the introduction of BI technology in healthcare environments.

Healthcare organizations typically store how their processes should be performed, especially those that represent complex routine jobs involving multiple people and organizational units. In the context of BI, medical processes are focused on activities and work practices in necessary health services provided (medical and nursing treatments) for the proper operation of a healthcare organization.

Thus, intelligent technologies can be considered facilitators of the management, storage, analysis, and visualization, but they also ensure access to large amounts of data in the context of BI. For such, a wide variety of technologies such as expert systems, online analytical processing, data mining (DM) and knowledge extraction are used in the development of a BI system in the healthcare sector. These technologies are required to provide an integrated view of internal and external data (data warehouse), which is considered to be the foundation of a BI system. On the other hand, BI systems also include software that provides tools for improving processes in companies, platforms for creating reports, graphical display, dimensional analysis, and DM models.[1][2][19]

Tool requirements in healthcare environments

The requirements of healthcare organizations for the implementation of a BI system are mainly for providing information to aid the decision-making process at a strategic level, with certain implications at the operational level. The BI technology uses historical and current data in order to visualize them through reports, graphs, and key performance indicators (KPIs), using analytical processing tools.

Thus, and keeping in mind the terms of clinical data, open-source applications must present a set of requirements to meet the needs of healthcare professionals. Some of these features are as follows[4][24][25][26][27][28]:

Performance: assesses whether the tool has good performance in query processing with a high volume of data. In the healthcare sector, it is important that the performance is good, since it is an area where decisions have a major impact on the lives of human beings and sometimes need to be made in a very short period of time.

Online Analytical Processing ad hoc queries: evaluate if the tool allows the user to have the freedom to define queries, which he considers appropriate in a given context. OLAP allows the users to perform ad hoc analyses on the data, considering multiple dimensions and providing the necessary information for an even more efficient decision-making process. This technique allows the analysis of the document’s history, and the use of operations such as the roll-up, drill-down, slice and dice, and pivot. In the healthcare sector, the process of analyzing historical clinical data is very important, since it allows the visualization of the patient and service evolution over time. Thus, this technique is essential in a BI system.

Architecture: assesses whether or not the tool implements a Data Warehouse (DW) and OLAP architectures with high scalability, i.e., capable of processing information evenly, even if the load is increasing.

Display of KPIs: assesses whether or not the tool provides visualization of the KPIs of the organization. These indicators can be clinical or management ones.

Plug-ins: assess whether or not the tool allows the development and use of plug-ins that add functionality to it.

Interactive visualization of data: assesses whether or not the tool allows interactivity between the user and dashboards, reports, and graphs. This is a very important characteristic since interactivity is appealing to the user, and also facilitates the understanding of the information demonstrated.

Documentation: assesses the quality of the documentation given by the tool. This feature is very important for the programmer who develops the application since the installation process is sometimes a complicated procedure that requires documentation.

Dashboards: assess whether or not the tool supports the development of dashboards, enabling the integration of graphics, tables, and other analyses such as OLAP and DM.

Navigation features: assess whether or not the tool enables the creation of reports, using roll-up, drill-down, slice and dice, and pivot operations.

ETL: steps of the BI process responsible for the extraction, transformation, and loading of data by creating procedures incorporated into the tool.

Connection to the database: it is very important that the BI tool enables a connection to be established to different databases so that it is possible to integrate information from different data sources. There are tools where the only possibility of connection is inherent to data visualization, others in which the connection can be made via the ETL process and via data display, and, lastly, those in which the connection is made only via the ETL process. In a hospital, this is also a key feature because, normally, these organizations have interoperable systems, and it is common to have different databases with clinical information. Thus, in order to facilitate the construction of the data warehouse (DW), we must choose a specific tool for the construction of the DW and, subsequently, a tool of BI is used to create OLAP cubes in order to visualize performance indicators.

Integration of dimensional model: evaluates whether or not the BI tool allows the integration of a DW dimensional model.

Open-source: assesses whether or not the tool presents a development model for which, besides being free, the source code is completely available for users to visualize, modify, and redistribute without restrictions placed by the owner of the product.

Export: assesses whether or not the tool allows export to other formats such as PDF, HTML, spreadsheets, and others.

Pervasive: assesses whether or not the free version of the tool provides a server that allows the development of a web application, which can be opened in a browser, or a mobile application that can be installed on different mobile devices and send alerts, and other pervasive data or healthcare characteristics.[27] If the tool has this feature, it is not necessary to install the application on all the computers in the organization, but only on a server, with all computers connected to the organization network able to access the web application. In a hospital, this feature is very important because, besides the reduction of costs in the application installation process, it also allows a reduction in the time spent, which is very important in the healthcare sector. Specifically, in the Centro Hospitalar do Porto (CHP), this aspect is also very important for the development of BI applications, as once the BI system is integrated into the AIDA it enables interoperability in all the constituents of the organization. Thus, pervasiveness is implemented in this context, so that the information is distributed to all the users of the organization and is not just focused at the top of the organizational pyramid.

Online help: assesses whether or not the tool provides online help resources.

Support for mobile devices: assesses whether or not the tool supports the use of mobile devices, which can be quite useful in a healthcare organization, in that health professionals are then able to access information by other means than a computer.

Data mining: evaluates whether or not the tool provides the ability to use data to predict chosen outcomes as clinical situations or behavior patterns.

Ease of use: assesses the ease with which a non-experienced user is able to identify and to find the tools’ features, as well as how easy it is to perform them.

Attractiveness: assesses the degree of a tool’s interface attractiveness.

Customization of the interface: identifies if the tool allows customization of the interface by the administrator.

User profile: verifies whether or not the tool allows the administrator to set hierarchies by assigning different permissions to different system users.

Real-time: maintains an approach to data analysis that allows users to access the application and the information in real time. This is an important feature in healthcare organizations because it is crucial that health professionals are able to access current data to support the decision-making process. It includes real-time data processing (ETL) and dashboard updates.

References

  1. ↑ 1.0 1.1 1.2 1.3 Janamanchi, B.; Katsamakas, E.; Raghupathi, W.; Gao, W. (2009). "The state and profile of open source software projects in health and medical informatics". International Journal of Medical Informatics 78 (7): 457-72. doi:10.1016/j.ijmedinf.2009.02.006. PMID 19321384. 
  2. ↑ 2.0 2.1 2.2 2.3 Waring, T.; Maddocks, P. (2005). "Open source software implementation in the UK public sector: Evidence from the field and implications for the future". International Journal of Information Management 25 (5): 411-28. doi:10.1016/j.ijinfomgt.2005.06.002. 
  3. ↑ 3.0 3.1 3.2 Muraina, I.D.; Ahmad, A. (30 July 2011). "Healthcare Business Intelligence: The Case of University's Health Center". Universiti Utara Malaysia. pp. 27. http://www.academia.edu/4465749/Healthcare_Business_Intelligence_The_Case_of_University_s_Health_Center. 
  4. ↑ 4.0 4.1 Bernardino, J. (2011). "Open Source Business Intelligence Platforms for Engineering Education" (PDF). Proceedings of the 1st World Engineering Education Flash Week, Lisbon 2011 2011: 693–698. http://www.sefi.be/wp-content/papers2011/T13/164.pdf. 
  5. ↑ Brandão, A.; Pereira, E.; Portela, F. et al. (2015). "Predicting the risk associated to pregnancy using data mining" (PDF). Proceedings of the 7th International Conference on Agents and Artificial Intelligence 2015. doi:10.5220/0005286805940601. http://repositorium.sdum.uminho.pt/bitstream/1822/36738/1/ICAART_2015_178_Andreia.pdf. 
  6. ↑ 6.0 6.1 "Canada Health Infoway". Canada Health Infoway, Inc. https://www.infoway-inforoute.ca/en/. Retrieved 10 July 2016. 
  7. ↑ "Qlik". QlikTech International AB. http://www.qlik.com/us/. Retrieved 12 October 2016. 
  8. ↑ "Jedox". Jedox AG. http://www.jedox.com/en/. Retrieved 12 October 2016. 
  9. ↑ "Jaspersoft". TIBCO Software, Inc. https://www.jaspersoft.com/. Retrieved 12 October 2016. 
  10. ↑ "Tableau Public". Tableau Software, Inc. https://public.tableau.com/s/. Retrieved 12 October 2016. 
  11. ↑ "SpagoBI". Engineering Ingegneria Informatica S.p.A. http://www.spagobi.org/. Retrieved 12 October 2016. 
  12. ↑ "Pentaho". Pentaho Corporation. http://www.pentaho.com/. Retrieved 12 October 2016. 
  13. ↑ 13.0 13.1 Completo, J.; Cruz, R.S.; Coheur, L.; Delgado, M. (2012). "Design and implementation of a data warehouse for benchmarking in clinical rehabilitation". Procedia Technology 5: 885–894. doi:10.1016/j.protcy.2012.09.098. 
  14. ↑ Côrte-Real, N. (November 2010). "Avaliação da Maturidade da Business Intelligence nas Organizações" (PDF). Universidade Nova de Lisboa. pp. 104. https://run.unl.pt/bitstream/10362/7477/1/TEGI0269.pdf. 
  15. ↑ Tereso, M.; Bernardino, J. (2011). "Open source business intelligence tools for SMEs". Proceedings of the 6th Iberian Conference on Information Systems and Technologies 2011. 
  16. ↑ "Care2X: The Open Source Hospital Information System". Care2x Team. http://www.care2x.org/. Retrieved 17 May 2016. 
  17. ↑ "OpenEHR: An Open Domain-Driven Platform for Developing Flexible e-Health Systems". openEHR Foundation. http://www.openehr.org/. Retrieved 03 May 2016. 
  18. ↑ Welch, W.P.; Bazarko, D.; Ritten, K. et al. (2007). "Electronic health records in four community physician practices: Impact on quality and cost of care". JAMIA 14 (3): 320–8. doi:10.1197/jamia.M2125. PMC PMC2244874. PMID 17329734. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2244874. 
  19. ↑ 19.0 19.1 Goulde, M.; Brown, E.; Rymer, J.; Hartzband, D. (March 2006). "Open Source Software: A Primer for Health Care Leaders". iHealth Reports. Forrester Consulting. pp. 31. http://www.chcf.org/publications/2006/03/open-source-software-a-primer-for-health-care-leaders. Retrieved 03 May 2016. 
  20. ↑ "SpagoBI for Areas". Engineering Ingegneria Informatica S.p.A. Archived from the original on 19 February 2014. https://web.archive.org/web/20140219065134/http://www.spagoworld.org/xwiki/bin/view/SpagoBI/ApplicationsAreas. Retrieved 01 November 2016. 
  21. ↑ "Pentaho Healthcare Solutions". SlideShare. Pentaho Corporation. 2012. http://pt.slideshare.net/Pentaho/pentaho-healthcare-solutions-presentation. Retrieved 10 July 2016. 
  22. ↑ Machado, J.; Alves, V.; Abelha, A.; Neves, J.M. (2007). "Ambient intelligence via multiagent systems in the medical arena". International Journal of Engineering Intelligent Systems 15: 151–158. 
  23. ↑ Peixoto, H.; Santos, M.; Abelha, A.; Machado, J.. "Intelligence in Interoperability with AIDA". In Chen, L.; Felfernig, A.; Liu, J.; Raś, Z.W.. Foundations of Intelligent Systems. Springer Berlin Heidelberg. pp. 264-273. doi:10.1007/978-3-642-34624-8_31. ISBN 9783642346248. 
  24. ↑ Portela, C.F.; Santos, M.F.; Marta, V.-B.. "A Pervasive Approach to a Real-Time Intelligent Decision Support System in Intensive Medicine". In Fred, A.; Dietz, J.L.G.; Liu, K.; Filipe, J.. Knowledge Discovery, Knowledge Engineering and Knowledge Management. Springer Berlin Heidelberg. pp. 368–381. doi:10.1007/978-3-642-29764-9_25. ISBN 9783642297649. 
  25. ↑ Lapa, J.; Bernardino, J.; Figueiredo, A. (2014). "A comparative analysis of open source business intelligence platforms". Proceedings of the International Conference on Information Systems and Design of Communication 2014: 86–92. doi:10.1145/2618168.2618182. 
  26. ↑ Sá, T.; Mateus, A.; Iwanaga, M. et al. (2011). "Uma Análise Comparativa entre as Ferramentas OLAP como Apoio a Soluções de BI nas Empresas" (PDF). PUC-Rio. http://www.infobrasil.inf.br/userfiles/15-S1-1-97127-Uma%20An%C3%A1lise%20Comparativa___.pdf. Retrieved 11 October 2016. 
  27. ↑ 27.0 27.1 Orwat, C.; Graefe, A.; Faulwasser, T. (2008). "Towards pervasive computing in health care - a literature review". BMC Medical Informatics and Decision Making 8: 26. doi:10.1186/1472-6947-8-26. PMC PMC2467411. PMID 18565221. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2467411. 
  28. ↑ Portela, C.F.; Santos, M.F.; Silva, Á. et al.. "Enabling a Pervasive Approach for Intelligent Decision Support in Critical Health Care". In Cruz-Cunha, M.M.; Varajão, J.; Powell, P.; Martinho, R.. ENTERprise Information Systems. Springer Berlin Heidelberg. pp. 233–243. doi:10.1007/978-3-642-24352-3_25. ISBN 9783642243523. 

Notes

This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added. The SpagoBI website had changed not long after publication, and archived URLs had to be used in a few cases.

Retrieved from "https://www.limswiki.org/index.php?title=Journal:A_benchmarking_analysis_of_open-source_business_intelligence_tools_in_healthcare_environments&oldid=28279"