View source for Journal:A benchmarking analysis of open-source business intelligence tools in healthcare environments

{{Infobox journal article
|name         = 
|image        = 
|alt          = <!-- Alternative text for images -->
|caption      = 
|title_full   = A benchmarking analysis of open-source business intelligence tools in healthcare environments
|journal      = ''Information''
|authors      = Brandão, Andreia; Pereira, Eliana; Esteves, Marisa; Portela, Filipe; Santos, Manuel F.; Abelha, António; Machado, José
|affiliations = University of Minho, Instituto Politécnico do Porto
|contact      = Tel.: +351-967-562-540
|editors      = Susilo, Willy
|pub_year     = 2016
|vol_iss      = '''7''' (4)
|pages        = 57
|doi          = [http://doi.org/10.3390/info7040057 10.3390/info7040057]
|issn         = 2078-2489
|license      = [http://creativecommons.org/licenses/by/4.0/ Creative Commons Attribution 4.0 International]
|website      = [http://www.mdpi.com/2078-2489/7/4/57/htm http://www.mdpi.com/2078-2489/7/4/57/htm]
|download     = [http://www.mdpi.com/2078-2489/7/4/57/pdf http://www.mdpi.com/2078-2489/7/4/57/pdf] (PDF)
}}
{{ombox	
| type      = content	
| style    = width: 500px;	
| text      = This article should not be considered complete until this message box has been removed. This is a work in progress.	
}}
==Abstract==
In recent years, a wide range of business intelligence (BI) technologies have been applied to different areas in order to support the decision-making process. BI enables the extraction of knowledge from the data stored. The healthcare industry is no exception, and so BI applications have been under investigation across multiple units of different institutions. Thus, in this article, we intend to analyze some open-source/free BI tools on the market and their applicability in the clinical sphere, taking into consideration the general characteristics of the [[Clinical laboratory|clinical environment]]. For this purpose, six BI tools were selected, analyzed, and tested in a practical environment. Then, a comparison metric and a ranking were defined for the tested applications in order to choose the one that best applies to the extraction of useful knowledge and clinical data in a healthcare environment. Finally, a pervasive BI platform was developed using a real case in order to prove the tool's viability.

'''Keywords''': business intelligence; open-source; healthcare; benchmarking

==Introduction==
Increasingly, the healthcare sector is looking for computer applications to support the daily practice of health professionals. [[Category:Open-source software|Open-source software]] is particularly desirable in this sector as, besides being free, it has a source code that is fully available to users for viewing, reading, modification, and redistribution, without the restrictions of ownership of the product (unlike free software, which only allows its use without charge).

Open-source software differs fundamentally from an ownership model in terms of the development process and the product licenses. All open-source applications are licensed by an open-source license, which gives the user the right to use the software, access and modify the source-code, and redistribute the software for free. This type of software is very popular due to its many advantages, since it promises to accelerate the diffusion of information technology (IT) solutions in healthcare. Thereby, it can contribute to reduced development costs.<ref name="JanamanchiTheState09">{{cite journal |title=The state and profile of open source software projects in health and medical informatics |journal=International Journal of Medical Informatics |author=Janamanchi, B.; Katsamakas, E.; Raghupathi, W.; Gao, W. |volume=78 |issue=7 |pages=457-72 |year=2009 |doi=10.1016/j.ijmedinf.2009.02.006 |pmid=19321384}}</ref><ref name="WaringOpen05">{{cite journal |title=Open source software implementation in the UK public sector: Evidence from the field and implications for the future |journal=International Journal of Information Management |author=Waring, T.; Maddocks, P. |volume=25 |issue=5 |pages=411-28 |year=2005 |doi=10.1016/j.ijinfomgt.2005.06.002}}</ref>

One the other hand, the choice of a free license software in [[Health informatics|health and medical informatics]] is important because it determines the user’s rights and can influence the developers’ willingness to participate in a project, the quality of the product, and the willingness of users to adopt certain applications.<ref name="JanamanchiTheState09" /><ref name="WaringOpen05" />

In terms of costs, organizations can save on licensing fees and reduce expenditures on specific computer hardware. However, organizations need to welcome and train specialized collaborators in the adoption of open-source solutions. This type of situation has the hidden costs of highly skilled employees, implementation, maintenance, and a support process, which may lead to adopting proprietary solutions.<ref name="JanamanchiTheState09" /><ref name="WaringOpen05" />

The acquisition of a business intelligence (BI) tool and its implementation are quite advantageous for the health organization. Extending their use can have a positive global impact. However, the healthcare environment has some particularities that a BI solution should be prepared to answer. For example, the BI system could lead to resource optimization in various departments; it will improve the clinical condition of the patient through efficient diagnosis and the identification and application of the best practice protocols for treatment, among others.<ref name="MurainaHealthcare11">{{cite web |url=http://www.academia.edu/4465749/Healthcare_Business_Intelligence_The_Case_of_University_s_Health_Center |title=Healthcare Business Intelligence: The Case of University's Health Center |author=Muraina, I.D.; Ahmad, A. |publisher=Universiti Utara Malaysia |date=30 July 2011 |pages=27}}</ref>

In order to help decision-makers make the best choices, a benchmarking study of BI tools focused on the healthcare environment was performed. After a thorough review of the literature, it was determined which tools were being used in this study. These tools were selected based on their good performance in several areas, such as management, healthcare, or retail.<ref name="BernardinoOpen11">{{cite journal |title=Open Source Business Intelligence Platforms for Engineering Education |journal=Proceedings of the 1st World Engineering Education Flash Week, Lisbon 2011 |author=Bernardino, J. |volume=2011 |pages=693–698 |year=2011 |url=http://www.sefi.be/wp-content/papers2011/T13/164.pdf |format=PDF}}</ref><ref name="BrandãoPredicting15">{{cite journal |title=Predicting the risk associated to pregnancy using data mining |journal=Proceedings of the 7th International Conference on Agents and Artificial Intelligence |author=Brandão, A.; Pereira, E.; Portela, F. et al. |volume=2015 |year=2015 |doi=10.5220/0005286805940601 |url=http://repositorium.sdum.uminho.pt/bitstream/1822/36738/1/ICAART_2015_178_Andreia.pdf |format=PDF}}</ref><ref name="CHI">{{cite web |url=https://www.infoway-inforoute.ca/en/ |title=Canada Health Infoway |publisher=Canada Health Infoway, Inc |accessdate=10 July 2016}}</ref> Thus, the tools selected for this study were QlikView<ref name="Qlik">{{cite web |url=http://www.qlik.com/us/ |title=Qlik |publisher=QlikTech International AB |accessdate=12 October 2016}}</ref>, Palo BI Suite<ref name="Jedox">{{cite web |url=http://www.jedox.com/en/ |title=Jedox |publisher=Jedox AG |accessdate=12 October 2016}}</ref>, Jaspersoft BI<ref name="Jaspersoft">{{cite web |url=https://www.jaspersoft.com/ |title=Jaspersoft |publisher=TIBCO Software, Inc |accessdate=12 October 2016}}</ref>, Tableau Public<ref name="TableauPublic">{{cite web |url=https://public.tableau.com/s/ |title=Tableau Public |publisher=Tableau Software, Inc |accessdate=12 October 2016}}</ref>, [[SpagoBI]]<ref name="SpagoBI">{{cite web |url=http://www.spagobi.org/ |title=SpagoBI |publisher=Engineering Ingegneria Informatica S.p.A |accessdate=12 October 2016}}</ref>, and Pentaho BI Suite.<ref name="Pentaho">{{cite web |url=http://www.pentaho.com/ |title=Pentaho |publisher=Pentaho Corporation |accessdate=12 October 2016}}</ref>

The analysis of this type of software emerged during a project that included the development of a BI platform in maternity care to visualize clinician and management indicators, as well as integrating data mining (DM) predictive models. All the tools were tested using real data provided by the Centro Hospital do Porto (CHP) and Centro Materno Infantil do Norte (CMIN). With this study, it is intended to select the BI tool that best suits the healthcare sector by using a practical case.

Besides the introduction, this article includes eight sections. The second section provides an approach to the background and related work, in which the concept of business intelligence is addressed and an introduction to the case study is presented. The third section is concerned with the application of BI tools in healthcare environments, followed by the requirements considered essential in these tools, in the section "Tool Requirements in Healthcare Environments." Thereafter, the next section addresses in more detail the tools selected; and the section "Business Intelligence Tools — Assessment Process" discusses our approach to the case study. Finally, the last sections correspond to the results, discussion, and conclusion.

==Background and related work==
===Business intelligence===
Business intelligence (BI) is the transformation of [[information]] stored in knowledge, enabling us to provide adequate information to a particular user at the appropriate time in order to support the decision-making process in real time. Thus, BI integrates a set of tools and technologies that enables the collection, integration, analysis, and visualization of data.
For the implementation of a BI platform, it is necessary to perform some intermediate steps that are common in the development of this type of software tool, such as the construction of a data warehouse (DW). In this case, the Kimball methodology was chosen in order to design as well as develop and deploy the DW.<ref name="CompletoDesign12">{{cite journal |title=Design and implementation of a data warehouse for benchmarking in clinical rehabilitation |journal=Procedia Technology |author=Completo, J.; Cruz, R.S.; Coheur, L.; Delgado, M. |volume=5 |pages=885–894 |year=2012 |doi=10.1016/j.protcy.2012.09.098}}</ref> Thus, some of these steps are considered crucial for the successful implementation of a BI system, such as: tasks planning and expected results; defining the architecture that aims to follow the BI system; the selection and installation of the most appropriate BI tool; building the data warehouse dimensional model; the extraction, transformation and loading (ETL) process; and, finally, the development of the BI application.<ref name="CompletoDesign12" />

Thus, in order to develop a BI application, initially, it is necessary to choose the software that is most appropriate for achieving the desired outcomes. Thereby, it is necessary to undertake an analysis of most of the software available and choose the one type that provides the necessary and desired resources.<ref name="MurainaHealthcare11" /><ref name="Côrte-RealAvalia10">{{cite web |url=https://run.unl.pt/bitstream/10362/7477/1/TEGI0269.pdf |format=PDF |title=Avaliação da Maturidade da Business Intelligence nas Organizações |author=Côrte-Real, N. |publisher=Universidade Nova de Lisboa |date=November 2010 |pages=104}}</ref><ref name="TeresoOpen11">{{cite journal |title=Open source business intelligence tools for SMEs |journal=Proceedings of the 6th Iberian Conference on Information Systems and Technologies |author=Tereso, M.; Bernardino, J. |volume=2011 |year=2011}}</ref>

===Examples of application of BI tools in healthcare environments===
With the urgency of acquiring medical informatics applications, open-source software is receiving increased attention from the healthcare industry. For example, the open-source project [[Care2X]] — composed of a [[hospital information system]], practice management, a central data server, and a health exchange protocol — is under development in Europe. Care2X was developed in order to overcome integration problems in multiple incompatible network programs. It is possible to integrate almost any type of service, system, department, process, data, or communication of a [[hospital]]. Care2X supports the clinical workflow, incorporating diagnosis-related groups (DRG), as well as scheduling and electronic prescribing modules.<ref name="Care2X">{{cite web |url=http://www.care2x.org/ |title=Care2X: The Open Source Hospital Information System |publisher=Care2x Team |accessdate=17 May 2016}}</ref>

[[openEHR]] corresponds to another case, which is sponsored by the openEHR Foundation. It promotes the "development of an open, interoperable health computing platform, of which the major component is clinically effective and interoperable [[electronic health record]]s (EHRs)."<ref name="openEHR">{{cite web |url=http://www.openehr.org/ |title=OpenEHR: An Open Domain-Driven Platform for Developing Flexible e-Health Systems |publisher=openEHR Foundation |accessdate=03 May 2016}}</ref>

[[Canada Health Infoway]], established by Canadian federal and provincial grants, began an open-source initiative in 2005 to develop software that hospitals and developers could use to ensure the secure exchange of medical records of patients between various entities.<ref name="CHI" /> Thereby, these initiatives suggest that open-source is a viable way of developing applications in healthcare.<ref name="WelchElectronic07">{{cite journal |title=Electronic health records in four community physician practices: Impact on quality and cost of care |journal=JAMIA |author=Welch, W.P.; Bazarko, D.; Ritten, K. et al. |volume=14 |issue=3 |pages=320–8 |year=2007 |doi=10.1197/jamia.M2125 |pmid=17329734 |pmc=PMC2244874}}</ref><ref name="GouldeOpen06">{{cite web |url=http://www.chcf.org/publications/2006/03/open-source-software-a-primer-for-health-care-leaders |title=Open Source Software: A Primer for Health Care Leaders |work=iHealth Reports |author=Goulde, M.; Brown, E.; Rymer, J.; Hartzband, D. |publisher=Forrester Consulting |pages=31 |date=March 2006 |accessdate=03 May 2016}}</ref>

In addition, it is noteworthy that currently there are already a few applications developed based on open-source tools, implemented in healthcare organizations, such as:

* Turin ASL 3, which is a system developed in conjunction with the open-source SpagoBI that allows the assignment of permissions for use according to the different types of users. It provides analytical documents, enables the data visualization, and also allows the use of the online analytical processing (OLAP) technology. This solution is implemented in local healthcare institutions and the Italian National Health Service. Turin ASL 3 was born in 1995 and is implemented in two hospitals in the city of Turin (Amadeo di Savoia and Maria Vittoria), Italy.<ref name="SpagoAreasArch">{{cite web |url=http://www.spagoworld.org/xwiki/bin/view/SpagoBI/ApplicationsAreas |archiveurl=https://web.archive.org/web/20140219065134/http://www.spagoworld.org/xwiki/bin/view/SpagoBI/ApplicationsAreas |title=SpagoBI for Areas |publisher=Engineering Ingegneria Informatica S.p.A |archivedate=19 February 2014 |accessdate=01 November 2016}}</ref>

* St Antonius is an application developed by Pentaho BI Suite, built in order to analyze the waiting times of patients. The main advantages of this application are the improvement of operational efficiency, elimination of costs resulting from the creation of manual reporting, behavioral analysis, and the identification of patterns and risk analysis. This application has been implemented in the St. Antonius Hospital, located in Nieuwegein, Netherlands.<ref name="SSPentaho">{{cite web |url=http://pt.slideshare.net/Pentaho/pentaho-healthcare-solutions-presentation |title=Pentaho Healthcare Solutions |work=SlideShare |publisher=Pentaho Corporation |date=2012 |accessdate=10 July 2016}}</ref>

===Case study - Context===
The analysis of business intelligence open-source software was one of the steps of a process conducted in order to develop a BI platform to support decision-making in maternity care in Centro Materno Infantil do Norte (CMIN). CMIN is one of the constituents of the Centro Hospitalar do Porto (CHP), along with the Hospital Santo António (HSA) and Hospital Joaquim Urbano (HJU).

The BI platform is directed at the modules of Gynecology and Obstetrics (GO) and Voluntary Interruption of Pregnancy (VIP) since these modules are lacking decision support systems. This platform aims to visualize the knowledge extracted from the data stored in information systems in CMIN, through their representation in tables, charts, and tables, among others, but also by integrating DM predictive models.

Some of the VIP key performance indicators (KPIs) that health professionals have interest in include:

* characterization of the patient group by number of pregnancies and by date;
* characterization of the patient group by number of children and by date;
* characterization of the patient group by number of previous VIP experiences and by date;
* characterization of the patient group concerning the revision consultation for date; and
* characterization of the patient group based on contraception early in the process by date.

All the information recorded in the CHP is stored in different systems, taking as an example the nursing support system (SAPE), where a portion of the data recorded in the VIP module is stored, and the electronic health record (EHR), where all the patient data are stored.

The Medical Information Integration, Dissemination and Storing Agency (AIDA) guarantees the interoperability of these systems. AIDA is a system of intelligent agents that allows communication between different information systems in CHP.<ref name="MachadoAmbient07">{{cite journal |title=Ambient intelligence via multiagent systems in the medical arena |journal=International Journal of Engineering Intelligent Systems |author=Machado, J.; Alves, V.; Abelha, A.; Neves, J.M. |volume=15 |pages=151–158 |year=2007}}</ref><ref name="PeixotoIntell12">{{cite book |chapter=Intelligence in Interoperability with AIDA |title=Foundations of Intelligent Systems |author=Peixoto, H.; Santos, M.; Abelha, A.; Machado, J. |editor=Chen, L.; Felfernig, A.; Liu, J.; Raś, Z.W. |publisher=Springer Berlin Heidelberg |pages=264-273 |isbn=9783642346248 |doi=10.1007/978-3-642-34624-8_31}}</ref>

==Applications of open-source BI tools in healthcare==
With the computerization of clinical processes in healthcare organizations, the storage of clinical data in databases has been increasing exponentially. However, a lack of technology to gather, analyze, and distribute the most relevant information makes these organizations rich in data but extremely poor in information.<ref name="MurainaHealthcare11" /> Nonetheless, forward-thinking healthcare organizations are aware that the data and its treatment through the business intelligence (BI) technology are essential for an informed and accurate decision-making process, as well as necessary to improve services and ensure the future of these organizations.

Cases of adoption of open-source BI tools by healthcare organizations are scarce. However, the great benefits arising from its implementation in other areas have led to the introduction of BI technology in healthcare environments.

Healthcare organizations typically store how their processes should be performed, especially those that represent complex routine jobs involving multiple people and organizational units. In the context of BI, medical processes are focused on activities and work practices in necessary health services provided (medical and nursing treatments) for the proper operation of a healthcare organization.

Thus, intelligent technologies can be considered facilitators of the management, storage, analysis, and visualization, but they also ensure access to large amounts of data in the context of BI. For such, a wide variety of technologies such as [[expert system]]s, online analytical processing, data mining (DM) and knowledge extraction are used in the development of a BI system in the healthcare sector. These technologies are required to provide an integrated view of internal and external data (data warehouse), which is considered to be the foundation of a BI system. On the other hand, BI systems also include software that provides tools for improving processes in companies, platforms for creating reports, graphical display, dimensional analysis, and DM models.<ref name="JanamanchiTheState09" /><ref name="WaringOpen05" /><ref name="GouldeOpen06" />

==Tool requirements in healthcare environments==
The requirements of healthcare organizations for the implementation of a BI system are mainly for providing information to aid the decision-making process at a strategic level, with certain implications at the operational level. The BI technology uses historical and current data in order to visualize them through reports, graphs, and key performance indicators (KPIs), using analytical processing tools.

Thus, and keeping in mind the terms of clinical data, open-source applications must present a set of requirements to meet the needs of healthcare professionals. Some of these features are as follows<ref name="BernardinoOpen11" /><ref name="PortelaAPerv13">{{cite book |chapter=A Pervasive Approach to a Real-Time Intelligent Decision Support System in Intensive Medicine |title=Knowledge Discovery, Knowledge Engineering and Knowledge Management |author=Portela, C.F.; Santos, M.F.; Marta, V.-B. |editor=Fred, A.; Dietz, J.L.G.; Liu, K.; Filipe, J. |publisher=Springer Berlin Heidelberg |pages=368–381 |isbn=9783642297649 |doi=10.1007/978-3-642-29764-9_25}}</ref><ref name="LapaAComp14">{{cite journal |title=A comparative analysis of open source business intelligence platforms |journal=Proceedings of the International Conference on Information Systems and Design of Communication |author=Lapa, J.; Bernardino, J.; Figueiredo, A. |volume=2014 |pages=86–92 |year=2014 |doi=10.1145/2618168.2618182}}</ref><ref name="SáUmaAn11">{{cite web |url=http://www.infobrasil.inf.br/userfiles/15-S1-1-97127-Uma%20An%C3%A1lise%20Comparativa___.pdf |format=PDF |title=Uma Análise Comparativa entre as Ferramentas OLAP como Apoio a Soluções de BI nas Empresas |author=Sá, T.; Mateus, A.; Iwanaga, M. et al. |publisher=PUC-Rio |date=2011 |accessdate=11 October 2016}}</ref><ref name="OrwatTowards08">{{cite journal |title=Towards pervasive computing in health care - a literature review |journal=BMC Medical Informatics and Decision Making |author=Orwat, C.; Graefe, A.; Faulwasser, T. |volume=8 |pages=26 |year=2008 |doi=10.1186/1472-6947-8-26 |pmid=18565221 |pmc=PMC2467411}}</ref><ref name="PortelaEnabling11">{{cite book |chapter=Enabling a Pervasive Approach for Intelligent Decision Support in Critical Health Care |title=ENTERprise Information Systems |author=Portela, C.F.; Santos, M.F.; Silva, Á. et al. |editor=Cruz-Cunha, M.M.; Varajão, J.; Powell, P.; Martinho, R. |publisher=Springer Berlin Heidelberg |pages=233–243 |isbn=9783642243523 |doi=10.1007/978-3-642-24352-3_25}}</ref>:

* '''Performance''': assesses whether the tool has good performance in query processing with a high volume of data. In the healthcare sector, it is important that the performance is good, since it is an area where decisions have a major impact on the lives of human beings and sometimes need to be made in a very short period of time.

* '''Online Analytical Processing ad hoc queries''': evaluate if the tool allows the user to have the freedom to define queries, which he considers appropriate in a given context. OLAP allows the users to perform ad hoc analyses on the data, considering multiple dimensions and providing the necessary information for an even more efficient decision-making process. This technique allows the analysis of the document’s history, and the use of operations such as the roll-up, drill-down, slice and dice, and pivot. In the healthcare sector, the process of analyzing historical clinical data is very important, since it allows the visualization of the patient and service evolution over time. Thus, this technique is essential in a BI system.

* '''Architecture''': assesses whether or not the tool implements a Data Warehouse (DW) and OLAP architectures with high scalability, i.e., capable of processing information evenly, even if the load is increasing.

* '''Display of KPIs''': assesses whether or not the tool provides visualization of the KPIs of the organization. These indicators can be clinical or management ones.

* '''Plug-ins''': assess whether or not the tool allows the development and use of plug-ins that add functionality to it.

* '''Interactive visualization of data''': assesses whether or not the tool allows interactivity between the user and dashboards, reports, and graphs. This is a very important characteristic since interactivity is appealing to the user, and also facilitates the understanding of the information demonstrated.

* '''Documentation''': assesses the quality of the documentation given by the tool. This feature is very important for the programmer who develops the application since the installation process is sometimes a complicated procedure that requires documentation.

* '''Dashboards''': assess whether or not the tool supports the development of dashboards, enabling the integration of graphics, tables, and other analyses such as OLAP and DM.

* '''Navigation features''': assess whether or not the tool enables the creation of reports, using roll-up, drill-down, slice and dice, and pivot operations.

* '''ETL''': steps of the BI process responsible for the extraction, transformation, and loading of data by creating procedures incorporated into the tool.

* '''Connection to the database''': it is very important that the BI tool enables a connection to be established to different databases so that it is possible to integrate information from different data sources. There are tools where the only possibility of connection is inherent to data visualization, others in which the connection can be made via the ETL process and via data display, and, lastly, those in which the connection is made only via the ETL process. In a hospital, this is also a key feature because, normally, these organizations have interoperable systems, and it is common to have different databases with clinical information. Thus, in order to facilitate the construction of the data warehouse (DW), we must choose a specific tool for the construction of the DW and, subsequently, a tool of BI is used to create OLAP cubes in order to visualize performance indicators.

* '''Integration of dimensional model''': evaluates whether or not the BI tool allows the integration of a DW dimensional model.

* '''Open-source''': assesses whether or not the tool presents a development model for which, besides being free, the source code is completely available for users to visualize, modify, and redistribute without restrictions placed by the owner of the product.

* '''Export''': assesses whether or not the tool allows export to other formats such as PDF, HTML, spreadsheets, and others.

* '''Pervasive''': assesses whether or not the free version of the tool provides a server that allows the development of a web application, which can be opened in a browser, or a mobile application that can be installed on different mobile devices and send alerts, and other pervasive data or healthcare characteristics.<ref name="OrwatTowards08" /> If the tool has this feature, it is not necessary to install the application on all the computers in the organization, but only on a server, with all computers connected to the organization network able to access the web application. In a hospital, this feature is very important because, besides the reduction of costs in the application installation process, it also allows a reduction in the time spent, which is very important in the healthcare sector. Specifically, in the Centro Hospitalar do Porto (CHP), this aspect is also very important for the development of BI applications, as once the BI system is integrated into the AIDA it enables interoperability in all the constituents of the organization. Thus, pervasiveness is implemented in this context, so that the information is distributed to all the users of the organization and is not just focused at the top of the organizational pyramid.

* '''Online help''': assesses whether or not the tool provides online help resources.

* '''Support for mobile devices''': assesses whether or not the tool supports the use of mobile devices, which can be quite useful in a healthcare organization, in that health professionals are then able to access information by other means than a computer.

* '''Data mining''': evaluates whether or not the tool provides the ability to use data to predict chosen outcomes as clinical situations or behavior patterns.

* '''Ease of use''': assesses the ease with which a non-experienced user is able to identify and to find the tools’ features, as well as how easy it is to perform them.

* '''Attractiveness''': assesses the degree of a tool’s interface attractiveness.

* '''Customization of the interface''': identifies if the tool allows customization of the interface by the administrator.

* '''User profile''': verifies whether or not the tool allows the administrator to set hierarchies by assigning different permissions to different system users.

* '''Real-time''': maintains an approach to data analysis that allows users to access the application and the information in real time. This is an important feature in healthcare organizations because it is crucial that health professionals are able to access current data to support the decision-making process. It includes real-time data processing (ETL) and dashboard updates.

==Business intelligence tools — Benchmarking==
The study was based only on the analysis of open-source solutions. To achieve this goal, some works were analyzed using scientific databases. During this process the most used open-source solutions were identified. Then the top six solutions (QlikView, Palo, Jaspersoft, Tableau, Spago, and Pentaho) were selected to be explored.

In this study, benchmarking was used in order to compare tools and define evaluation metrics, focusing on the healthcare industry. In the following subsections, an overview of the business intelligence (BI) tools tested during this study are presented, based on the experiments made, as well as an extended analysis or literature review, of each.<ref name="WaringOpen05" /><ref name="BernardinoOpen11" /><ref name="SpagoAreasArch" /><ref name="LapaAComp14" /><ref name="SáUmaAn11" /><ref name="JedoxWhite09Arch">{{cite web |url=http://www.altavia.com/avcfiles/jedox_whitepaper.pdf |archiveurl=https://web.archive.org/web/20150319163424/http://www.altavia.com/avcfiles/jedox_whitepaper.pdf |format=PDF |title=Jedox Whitepaper |publisher=Jedox AG |date=2009 |archivedate=19 March 2015 |accessdate=02 November 2016}}</ref><ref name="ArriagaWeb08">{{cite web |url=http://www4.di.uminho.pt/~prh/uce15-0809/g15.pdf |format=PDF |title=Web OLAP, Exploração de Sistemas OLAP via Web |author=Arriaga, A.; Marques, P. |publisher=University of Minho |date=2008 |accessdate=12 October 2016}}</ref><ref name="MSMicroStrategy11">{{cite web |url=https://www.ndm.net/bi/pdf/MicroStrategy-9-vs-QlikTech-11.pdf |format=PDF |title=MicroStrategy 9 vs. QlikTech (QlikView 11) |publisher=MicroStrategy Incorporated |date=2011 |pages=39 |accessdate=12 October 2016}}</ref><ref name="HanleyQlik10Arch">{{cite web |url=http://www.covaligroup.com/blog/bid/47883/QlikView-vs-OLAP |archiveurl=https://web.archive.org/web/20131003134201/http://www.covaligroup.com/blog/bid/47883/QlikView-vs-OLAP |title=QlikView vs OLAP |author=Hanley, C. |publisher=Covali Group |date=11 October 2010 |archivedate=03 October 2013 |accessdate=02 November 2016}}</ref><ref name="KamkolkarTableau15">{{cite web |url=http://www.tableau.com/sites/default/files/media/whitepapertableauforenterprise_0.pdf |format=PDF |title=Tableau for the Enterprise: An Overview for IT |author=Kamkolkar, N.; Fields, E.; Rueter, M. |publisher=Tableau Software, Inc |date=2015 |accessdate=12 October 2016}}</ref> All the BI tools were selected based on web-based studies, i.e., through a review of the literature it was determined which tools were to be used in this study.

===QlikView===

QlikView (QV) is BI software developed by the Swedish company QlikTech. Although this software is a proprietary product, the company provides a full development version of the software for free. Nonetheless, the company offers various licensing options by limiting the use of the software in accordance with the license acquired by the user.

A key feature of this software is that it is fairly simple to extract data from different sources by allowing connections through Open Database Connectivity (ODBC) and Object Linking and Embedding Database (OLEDB), which are communication interfaces between the operating system and the various databases. OLAP also allows several operations by facilitating the user navigation between different dimensions through ad hoc queries. Moreover, it also enables the creation of a friendly and flexible interface with charts, pivot tables, and statistical analyses.

Another important feature of the software is that it does not use a database management system (DBMS) as a storage tool. The system connects itself to the base via online transaction processing (OLTP), used only upon completion of the data loading process. Thus, the data are submitted to ETL processes and compressed in a file with extension .qvw, understood by QV. This QlikView file contains all the details required for data analysis, including the data itself, necessary to update the QV script file with new data from the data source; the layout information (folders, lists, charts, etc.); alerts, bookmarks, documents and reports; and information about access restrictions and the module macros.

Moreover, the distribution of information is facilitated and the analysis can be performed regardless of the original data location or network conditions. Thus, the possibility arises of the end user viewing the application in many ways; once the .qvw file is generated, it may be viewed on any machine with the data and reports produced. In addition, access can be from a browser, through an application server, according to the safety rules of the customer. However, this feature is only possible in the paid version of QV. In the free version, the final document can only be used on the machine where it was initially developed.

In conclusion, the advantages of QV are the ease of reporting by end users, since the ETL processes are being held, and the fact that reports can be produced with the inclusion of graphics and basic knowledge of aggregate functions (e.g., sums and counts).

===Palo BI Suite===
The Jedox products are completely open-source, ready to be installed and used. This software aims to minimize the amount of overhead, time, cost, and involvement of computer technologies in the creation and maintenance of BI solutions. Thus, users assume a leadership role through the creation and management of data in Palo BI Suite. The Community version of Palo BI Suite consists of the following components:

* Palo OLAP Server is a multidimensional OLAP in-memory (Molap). The data are stored and then organized into cubes, dimensions, elements, and element attributes. Compared with Rolap (Relational OLAP), the Molap has the potential to be 100 times faster. A simple Palo cube can contain data from multiple data sources, which simplifies the analysis of data from different data sources, for example, the comparison of actual and predicted data collection. Users can import data that rises incrementally, from any data source that Palo gets access to, and stores them in the structure defined by them. The way that the tables are listed in the database is not relevant, since this architecture is defined in the software itself.

* Palo Excel Add-in allows a connection to Excel via a centralized and multidimensional database such as the Palo OLAP server, making it a highly sophisticated BI tool.

* Palo Worksheet Server provides a complete reporting and analysis web-based system that can be maintained by the users. The reports can be created and published to the web, and a flexible and secure content management is provided. Since it is quite similar to Excel, users can easily adapt to this tool by creating reports in the same way as in Excel. However, the resulting calculation sheets are automatically updated when the databases are changed, e.g., when new data is added to a dimension.

* Palo ETL Server supports the ETL process. Palo is equipped for the extraction of large quantities of data from a wide range of data sources. The ETL server is not limited to loading the Palo OLAP Server; it is also adapted to the specific needs of the import and export of data to and from Palo models.

===Jaspersoft BI===
The Jaspersoft BI is a tool, developed in 2001, in Java and Perl languages. This is an open-source tool available in two versions: the Community version and the Enterprise version. The version of Jaspersoft BI Community consists of six individual components:

* Jaspersoft iReport Designer is the report designer for JasperReports and JasperReports Server. It enables the creation of sophisticated layouts with graphics, images, and tables. It also allows access to data via JDBC, TableModels, JavaBeans, XML, Hibernate, CSV, and custom sources. Additionally, reports can be exported in PDF, RTF, XML, XLS, CSV, HTML, DOCX, or OpenOffice formats.

* Jaspersoft Studio has the same functionality as the Jaspersoft iReport Designer, differing only in that it is based on Eclipse.

* [[JasperReports Library]] is an open-source reporting engine, fully written in Java, that can use data from any data source and produce pixel-perfect documents that can be viewed, printed, or exported in a variety of document formats.

* JasperSoftReports Server is a reporting server allowing access to reports and analysis that can be incorporated into a web page or a mobile application, and providing real-time information, which is scheduled for a browser, mobile device, printer, or e-mail inbox in a great variety of file formats. It is optimized to share, protect, and centrally manage Jaspersoft reports and analytical views.

* Jaspersoft OLAP is a powerful environment for data analysis that can be accessed through an intuitive user interface, designed for the analysis of large volumes of datasets and to perform complex analytical queries. Consisting of an OLAP engine, this tool provides an interactive environment for users to perform slice and dice, pivot, and filter operations and to summarize data in real time through a web-based interface or MS Excel.

* Jaspersoft ETL is a simple deployment and running of the ETL process in many external systems. It is used to extract data from the transactional system, to create a data warehouse or data mart, and for creating reports or analysis.

===Tableau Public===
Tableau Public is web-based software and the open-source version of Tableau Desktop. This tool allows the creation of interactive visualizations and the option to embed them in a website, as well as publishing them on Tableau Public Gallery or sharing them in the Community Tableau Public. This open-source version differs from the paid version mainly in that the views cannot be saved locally. Thus, it is only possible to generate a code that can be embedded in any web page. However, this free version also offers several types of representation of data, such as graphs, tables, and maps.

On the other hand, Tableau Public also allows the use of various file formats, such as text files, spreadsheets, databases, and comma-separated files. When establishing a connection to a data source, this software identifies the role of each field, including whether the field contains dimensions and facts, i.e., it recognizes the dimensional model of the structure where the data is stored. Thus, when the data are selected to open the Tableau Public, the titles, columns, and rows of the structured data are all recognized, and only a simple drag-and-drop of these values is needed in order to create charts.

===SpagoBI===
The SpagoBI tool is open-source software, and there is only a single version, i.e., the community edition, a completely free version. It is a tool developed by Spago World and supported by an open-source community and consists of several modules:

* SpagoBI server corresponds to the main module of this software. It offers all the core and analytical capabilities of the application.

* SpagoBI studio is a development environment based on Eclipse. It allows the user to design and modify all the analysis documents such as reports, OLAP, dashboards, and DM. The interaction between this module and the SpagoBI server is possible due to the SpagoBI SDK module.

* SpagoBI Meta is a module oriented towards the management of metadata and search. The platform manages the metadata, allowing the user to edit and import from external tools such as ETL. This module enriches the knowledge base of metadata from SpagoBI server, so that they can be easily queried through available tools, such as OLAP.

* SpagoBI SDK is the specific tool used to integrate services provided by the server. Specifically, it is used by SpagoBI Studio so that the users can download/upload the analysis of documents to/from the server. This module allows the integration of documents, due to a wide range of services available through a web service, and the publishing of SpagoBI documents on an external portal.

* SpagoBI Applications is a collection of analytical models developed using SpagoBI. These models are developed taking into consideration the sector of the market and the purpose or the end product, such as analytical-specific components.

===Pentaho BI Suite===
Pentaho BI Suite software was developed by Pentaho Corporation in 2001, in the Java language, being the first BI platform to be released as an open-source alternative in the market. Pentaho offers two types of licenses: the Community Edition (CE), which is the open-source version, and the Enterprise Edition (EE), based on a subscription model.

The Pentaho BI Suite project comprises a set of products: BI platform (server), reporting, OLAP analysis, data integration (ETL), dashboards, and data mining.

The platform that integrates Pentaho is run entirely on the Java 2 Platform Enterprise Edition (J2EE), thus ensuring its scalability, integration, and portability. Also, it allows the connection to databases of JDBC, IBM DB2, Microsoft SQL Server, [[MySQL]], Oracle, [[PostgreSQL]], Firebird, and NCR Teradata types. Pentaho is structured into different modules, namely:

* Pentaho BI Platform provides various services to end users, such as subscriptions scheduling, reporting and integration tools, and incorporated centralized security.

* Pentaho Reporting allows the easy development of a report, enabling organizations to access, format, and distribute information. This module contains all the graphical features for the construction of reports as well as ad hoc queries.

* Pentaho Analysis provides an OLAP analysis, supporting the users in the decision-making process. The Pentaho Analysis facilitates the interactive exploration of information through the intersection of data, in addition to providing a complete integration with other services available in the Pentaho BI Suite, via plugins.

* Pentaho Data Integration is a powerful tool for ETL process using an innovative, metadata-driven approach.

* Community Edition Dashboard provides a graphical environment allowing users access to critical information essential to the understanding and optimization of organizational performance. Complete integration with Pentaho Reporting and Pentaho Analysis is possible.

* Weka Pentaho Data Mining enables a predictive analysis, providing information about hidden patterns and relationships between data, as well as performance indicators. This module provides a graphical interface for pre-processing of data, classification, regression, clustering, rules association, and visualization.

==Business intelligence tools — Assessment process==
In this section, a relationship is made between the features referenced as the requirements of a business intelligence (BI) platform in a healthcare environment with the BI tools previously chosen and analyzed based on web-based studies, i.e., through a review of the literature.

The assessment process was made in a group setting (10 people, five nurses and five IT specialists). A nurse was responsible for creating a team and providing us with the final observations. They performed two rounds of assessment and then provided us with the final assessment of each feature (group result). In order to maintain the anonymity of the results, we did not interfere in the assessment process. At the end, a meeting was held with the responsible nurse in order to understand the group’s opinions.

Thus, in Table 1, a comparison of the tools and the requirements is given. For the comparative analysis of the selected tools, a classification criterion based on the following scale has been chosen:

{| 
 | STYLE="vertical-align:top;"|
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="60%"
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" colspan="8"|'''Table 1.''' Comparative table of the BI open-source tools selected by features and group. (0 means unknown; 1 means absent; 2 means insufficient; 3 means sufficient; 4 means good and 5 means excellent.)
 |-
  ! style="padding-left:10px; padding-right:10px;" rowspan="2"|Features
  ! style="padding-left:10px; padding-right:10px;" colspan="6"|BI Open-Source Tools
  ! style="padding-left:10px; padding-right:10px;" rowspan="2"|Group
 |- 
  ! style="padding-left:10px; padding-right:10px;" |''Jaspersoft BI''
  ! style="padding-left:10px; padding-right:10px;" |''Palo BI Suite''
  ! style="padding-left:10px; padding-right:10px;" |''Pentaho BI Suite''
  ! style="padding-left:10px; padding-right:10px;" |''QlikView''
  ! style="padding-left:10px; padding-right:10px;" |''SpagoBI''
  ! style="padding-left:10px; padding-right:10px;" |''Tableau Public''
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Performance
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|D
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|OLAP Ad hoc Queries
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|B
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Architecture
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|D
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Display of KPIs
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|A
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Plug-ins
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3
  | style="background-color:white; padding-left:10px; padding-right:10px;"|D
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Interactive Visualization of Data
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|C
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Documentation
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3
  | style="background-color:white; padding-left:10px; padding-right:10px;"|F
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Dashboards
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|B
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Navigation Features
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|C
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|ETL
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|E
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Connection to the Database
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3
  | style="background-color:white; padding-left:10px; padding-right:10px;"|A
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Integration of Dimensional Model
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|E
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Open-source
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|D
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Export
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|C
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Pervasive
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|A
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Online Help
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|F
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Support for Mobile Devices
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3
  | style="background-color:white; padding-left:10px; padding-right:10px;"|C
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Data Mining
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3
  | style="background-color:white; padding-left:10px; padding-right:10px;"|2
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|B
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Ease of Use
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|F
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Attractiveness
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|3
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|C
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Customization of the Interface
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|F
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|User Profile
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|0
  | style="background-color:white; padding-left:10px; padding-right:10px;"|D
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Real-time
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|4
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|5
  | style="background-color:white; padding-left:10px; padding-right:10px;"|1
  | style="background-color:white; padding-left:10px; padding-right:10px;"|A
 |-
|}
|}

Thus, for each of the characteristics, each tool was rated. This rating reveals the tool satisfaction and compliance level in accordance with the characteristic analyzed. The score was based on the experimentation and critical view of the users.

On the other hand, after analyzing each of the characteristics for each tool, the requirements were grouped by a degree of similarity among them. Six groups were considered, where all the requirements were distributed, and a percentage was attributed according to their importance in a BI platform in a healthcare context.

It must be noted that all the scores attributed and the weightings assigned to each group were defined based on the critical opinion of a multidisciplinary team of health professionals, i.e., physicians and nurses from Centro Materno Infantil do Norte (CMIN), and IT professionals from Centro Hospitalar do Porto (CHP), including the authors of this paper. Thus, the first part of the evaluation was based on collecting opinions from several professionals involved directly or indirectly with the use of BI tools and, then, the critical analysis and assessment of all the feedback collected from the professionals by the authors involved in this study.

Thus, initially, the group of indispensable features (must have), whose importance cannot be measured, i.e., strictly necessary, was identified.

Then, a group was defined in which the requirements were targeted for the benefit of the administrator responsible for the construction of the BI platform. This group was assigned a percentage of 5%, because they do not have great importance in the health organization. It may have some value only to the programmer who manages the tool.

Another set group was the group whose characteristics are advantageous for the end-user of the BI platform. This group was assigned a percentage of 25%. It has a higher percentage assigned because it is important that the end-user is satisfied. Otherwise, this may cause the failure of the platform’s implementation.

On the other hand, the group with the highest percentage associated (30%) corresponds to the technologies that the tool incorporates, because the main focus of these tools are study and analysis functionalities. With the non-existence of features such as OLAP technologies or dashboards, a BI platform loses all its interest to IT and health professionals.

To some other important features, a percentage of 25% was allocated.

Finally, the last group created was associated with the processing of data and has a percentage of 15% associated. In this group, all the features included would be advantageous to incorporate in the tool; however, they do not have much influence on the success of the construction of the BI platform. For example, in the ETL process, if the tool could incorporate ETL procedures, it would have many benefits. However, the ETL may be performed using other tools.

The groups, associated features, and respective percentages are all shown in Table 2.

{| 
 | STYLE="vertical-align:top;"|
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="100%"
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" colspan="8"|'''Table 2.''' The groups, features associated, and respective percentages
 |-
  ! style="padding-left:10px; padding-right:10px;"|Group
  ! style="padding-left:10px; padding-right:10px;"|Characteristics
  ! style="padding-left:10px; padding-right:10px;"|Percentage
 |- 
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="4"|A - Must Have
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Pervasive
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="4"|-
 |- 
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Real-time
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Display of Key Performance Indicators
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Connection to the Database
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="3"|B - Technologies
  | style="background-color:white; padding-left:10px; padding-right:10px;"|OLAP Ad hoc Queries
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="3"|30%
 |- 
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Dashboards
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Data Mining
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="5"|C - End-User
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Interactive Visualization of Data
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="5"|25%
 |- 
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Navigation Features
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Export
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Support for Mobile Devices
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Attractiveness
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="5"|D - Other Important
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Performance
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="5"|25%
 |- 
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Architecture
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Plug-ins
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Open-source
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|User Profile
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="2"|E - Data Processing
  | style="background-color:white; padding-left:10px; padding-right:10px;"|ETL
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="2"|15%
 |- 
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Integration of Dimensional Model
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="4"|F - Administration
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Documentation
  | style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="4"|5%
 |- 
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Online Help
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Ease of Use
 |-
  | style="background-color:white; padding-left:10px; padding-right:10px;"|Customization of the Interface
 |-
|}
|}

==References==
{{Reflist|colwidth=30em}}

==Notes==
This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added. Several URLs have changed since publication, and archived URLs have been used in their place. What used to be citation 34 was removed as it was merely the URL of a search result on Google Scholar.

<!--Place all category tags here-->
[[Category:LIMSwiki journal articles (added in 2016)‎]]
[[Category:LIMSwiki journal articles (all)‎]]
[[Category:LIMSwiki journal articles on health informatics]]
[[Category:LIMSwiki journal articles on software‎‎]]