Difference between revisions of "User:Shawndouglas/sandbox/sublevel9"
Shawndouglas (talk | contribs) |
Shawndouglas (talk | contribs) |
||
Line 48: | Line 48: | ||
Figure 2 shows the key cost-based considerations of a basic LIMS installation project, with details for both on-premises and cloud-based systems. Depending on the vendor, there are potential additional costs. For example, any additional instrument interfacing costs may depend on the vendor's approach, the number and type of instruments in your lab, and so on. The same applies to connections to other software systems (e.g., accounting, statistical data systems, and others). Instrument and software integration costs aside, Figure 2 also brings up: | Figure 2 shows the key cost-based considerations of a basic LIMS installation project, with details for both on-premises and cloud-based systems. Depending on the vendor, there are potential additional costs. For example, any additional instrument interfacing costs may depend on the vendor's approach, the number and type of instruments in your lab, and so on. The same applies to connections to other software systems (e.g., accounting, statistical data systems, and others). Instrument and software integration costs aside, Figure 2 also brings up: | ||
* ''License, maintenance, and support costs'': Whether one-time or recurring, there will be a license fee for the software. And it will require scheduled maintenance and support for when things go wrong. | *''License, maintenance, and support costs'': Whether one-time or recurring, there will be a license fee for the software. And it will require scheduled maintenance and support for when things go wrong. | ||
* ''Acceptance and validation testing costs'': Unless the vendor includes this in their implementation package, you may have to budget for the necessary tests to ensure the system is meeting the needs and requirements of your laboratory and its stakeholders. | *''Acceptance and validation testing costs'': Unless the vendor includes this in their implementation package, you may have to budget for the necessary tests to ensure the system is meeting the needs and requirements of your laboratory and its stakeholders. | ||
* ''System and account configuration costs'': You could try to have your soon-to-be system administrator manually add user accounts, configure roles, and make other configuration changes, or it may make sense to have the vendor do the work for you in advance, for a price. | *''System and account configuration costs'': You could try to have your soon-to-be system administrator manually add user accounts, configure roles, and make other configuration changes, or it may make sense to have the vendor do the work for you in advance, for a price. | ||
* ''Data migration costs'': If your lab already has electronic data of some sort, it may be viewed as mission-critical, with a need to modify and migrate it to the new LIMS in an acceptable format. Such data migration projects can be time-consuming and costly, depending on the complexity and "dirtiness" of the data being imported. | *''Data migration costs'': If your lab already has electronic data of some sort, it may be viewed as mission-critical, with a need to modify and migrate it to the new LIMS in an acceptable format. Such data migration projects can be time-consuming and costly, depending on the complexity and "dirtiness" of the data being imported. | ||
* ''Networking costs'': Whether installed locally or in the cloud, some sort of networking is required (unless your a tiny lab with a single computer, running software locally). The client instances of the LIMS will need to electronically communicate with the server instance, or the web-based access screen will need to electronically communicate with the vendor hosting the LIMS in the cloud. This means local networking costs and/or a highly-dependable internet connection. | *''Networking costs'': Whether installed locally or in the cloud, some sort of networking is required (unless your a tiny lab with a single computer, running software locally). The client instances of the LIMS will need to electronically communicate with the server instance, or the web-based access screen will need to electronically communicate with the vendor hosting the LIMS in the cloud. This means local networking costs and/or a highly-dependable internet connection. | ||
* ''Supporting equipment and hardware costs'': From barcode readers and label printers to system terminals and document printers, the implementation of your LIMS may require supporting equipment that, in tandem with the LIMS, will improve workflows and expedite laboratory tasks. The LIMS introduces improved sample and specimen tracking using barcode labels and scanners, which benefits the lab, but is useless without the additional equipment. If the LIMS is to be hosted on-premises, IT hardware will also likely need to be acquired. | *''Supporting equipment and hardware costs'': From [[barcode]] readers and label printers to system terminals and document printers, the implementation of your LIMS may require supporting equipment that, in tandem with the LIMS, will improve workflows and expedite laboratory tasks. The LIMS introduces improved sample and specimen tracking using barcode labels and scanners, which benefits the lab, but is useless without the additional equipment. If the LIMS is to be hosted on-premises, IT hardware will also likely need to be acquired. | ||
* ''Training costs'': Some vendors may include a set amount of training with their implementation package, while others may not. Your lab is likely going to want to hit the ground running with the new LIMS, which means well-trained personnel, with full documentation and recorded training videos. As new personnel are on-boarded, your lab may not want to be the one responsible for training, opting to ask the vendor to run the employee through training. | *''Training costs'': Some vendors may include a set amount of training with their implementation package, while others may not. Your lab is likely going to want to hit the ground running with the new LIMS, which means well-trained personnel, with full documentation and recorded training videos. As new personnel are on-boarded, your lab may not want to be the one responsible for training, opting to ask the vendor to run the employee through training. | ||
* ''Project management costs'': Acquiring and implementing a LIMS is a project, full stop. The costs associated with project management may be handled internally, through normal hourly- or salary-based payment of personnel and managers for their organizational duties. However, some labs may not have the internal resources and experience to field such a project, requiring either a consultancy or even hiring the LIMS vendor to work with a lab stakeholder to oversee the project. | *''Project management costs'': Acquiring and implementing a LIMS is a project, full stop. The costs associated with project management may be handled internally, through normal hourly- or salary-based payment of personnel and managers for their organizational duties. However, some labs may not have the internal resources and experience to field such a project, requiring either a consultancy or even hiring the LIMS vendor to work with a lab stakeholder to oversee the project. | ||
Line 74: | Line 74: | ||
The economics behind the on-premises LIMS are generally different from those of the cloud-based LIMS. An on-premises system ultimately sees the system administrator taking responsibility for most or all aspects of the system. This includes: | The economics behind the on-premises LIMS are generally different from those of the cloud-based LIMS. An on-premises system ultimately sees the system administrator taking responsibility for most or all aspects of the system. This includes: | ||
* ''Hardware selection'': This includes the computer and associated components (e.g., printer, power backup, software backup, etc.). Additionally, the admin will need to find a place to put the computer and make arrangements with IT support for their services. | *''Hardware selection'': This includes the computer and associated components (e.g., printer, power backup, software backup, etc.). Additionally, the admin will need to find a place to put the computer and make arrangements with IT support for their services. | ||
* ''LIMS installation'': This includes installation of the operating system, LIMS software, and any underlying components (e.g., database software, support tools, etc.). This may be done alone or with the vendor's and IT services' assistance. | *''LIMS installation'': This includes installation of the operating system, LIMS software, and any underlying components (e.g., database software, support tools, etc.). This may be done alone or with the vendor's and IT services' assistance. | ||
* ''LIMS validation and testing'': The admin will be responsible for full system validation and testing. This significantly increases the workload of project management and support. | *''LIMS validation and testing'': The admin will be responsible for full system validation and testing. This significantly increases the workload of project management and support. | ||
The LIMS-specific software costs and process of using this type of LIMS implementation include the following: | The LIMS-specific software costs and process of using this type of LIMS implementation include the following: | ||
* Payment of the initial LIMS license and account setup fees is a one-time charge. These charges are usually based on the number of concurrent users of the system (sometimes referred to as “seats”). If you have six people in your lab, each can have their own login (meeting regulatory requirements), but only the specified number of concurrent users can access the software simultaneously. For example, with six accounts, and a license for two seats, two people can be working on the LIMS simultaneously. It may help to review several use cases internally, as well as their associated costs, to get a sense for how many accounts you need and can afford. Table 3 provides an example of a tabular view one might take with calculating license costs for several use cases. | *Payment of the initial LIMS license and account setup fees is a one-time charge. These charges are usually based on the number of concurrent users of the system (sometimes referred to as “seats”). If you have six people in your lab, each can have their own login (meeting regulatory requirements), but only the specified number of concurrent users can access the software simultaneously. For example, with six accounts, and a license for two seats, two people can be working on the LIMS simultaneously. It may help to review several use cases internally, as well as their associated costs, to get a sense for how many accounts you need and can afford. Table 3 provides an example of a tabular view one might take with calculating license costs for several use cases. | ||
{| | {| | ||
Line 89: | Line 89: | ||
|- | |- | ||
| style="background-color:white; padding-left:10px; padding-right:10px;" |'''1.''' | | style="background-color:white; padding-left:10px; padding-right:10px;" |'''1.''' | ||
| style="background-color:white; padding-left:10px; padding-right:10px; | | colspan="2" style="background-color:white; padding-left:10px; padding-right:10px;" |Number of people in the lab needing accounts: | ||
| style="background-color:white; padding-left:10px; padding-right:10px;" | | | style="background-color:white; padding-left:10px; padding-right:10px;" | | ||
|- | |- | ||
Line 140: | Line 140: | ||
|} | |} | ||
* Once the software has been installed and passes the necessary qualification testing, the next step is to configure the system to reflect your lab's requirements. This is more demanding than a cloud-based [[software as a service]] (SaaS) implementation since the project manager (PM) is responsible for all aspects of the work instead of being divided between the PM and the vendor; the vendor may provide guidance for some of this work based on their experience. This would include setting up test descriptions (including test and product limits), personnel data, relevant initial customer data, report formats, access controls, etc., whatever you define as needed to have a functioning system. This would not include instrument connections or connections to other systems. While those can be and are done, they usually entail an additional cost. Issues like that, and the details of their inclusion, are beyond the scope of this document, but you should be able to include them easily. The costs for instrument connections can vary widely depending on the devices' nature, use, and vendor’s support. | *Once the software has been installed and passes the necessary qualification testing, the next step is to configure the system to reflect your lab's requirements. This is more demanding than a cloud-based [[software as a service]] (SaaS) implementation since the project manager (PM) is responsible for all aspects of the work instead of being divided between the PM and the vendor; the vendor may provide guidance for some of this work based on their experience. This would include setting up test descriptions (including test and product limits), personnel data, relevant initial customer data, report formats, access controls, etc., whatever you define as needed to have a functioning system. This would not include instrument connections or connections to other systems. While those can be and are done, they usually entail an additional cost. Issues like that, and the details of their inclusion, are beyond the scope of this document, but you should be able to include them easily. The costs for instrument connections can vary widely depending on the devices' nature, use, and vendor’s support. | ||
* Once the system is set up, a system test and acceptance process begins as a part of a complete system acceptance/validation program. This compares the contents of the user requirement document to the functioning systems. This allows you to verify that the system works according to your needs and specifications. This is part of the validation process but not the entire process. | *Once the system is set up, a system test and acceptance process begins as a part of a complete system acceptance/validation program. This compares the contents of the user requirement document to the functioning systems. This allows you to verify that the system works according to your needs and specifications. This is part of the validation process but not the entire process. | ||
* Recurring charges include the annual license renewal fee, usually a fraction of the initial license cost. In addition, there would be hardware support costs, software maintenance fees (e.g., underlying components, including the operating system), backup systems annual fees (e.g., power, hardware, software), and IT support costs. Support can involve software and hardware vendors, facilities management (e.g., backup power), and IT support. | *Recurring charges include the annual license renewal fee, usually a fraction of the initial license cost. In addition, there would be hardware support costs, software maintenance fees (e.g., underlying components, including the operating system), backup systems annual fees (e.g., power, hardware, software), and IT support costs. Support can involve software and hardware vendors, facilities management (e.g., backup power), and IT support. | ||
Cloud-based SaaS LIMS, on the other hand, are hosted on the vendor's server and is ready to be accessed and used once it is initialized with your lab's specific details. No local software installation is needed. The costs and process of using this software implementation include: | Cloud-based SaaS LIMS, on the other hand, are hosted on the vendor's server and is ready to be accessed and used once it is initialized with your lab's specific details. No local software installation is needed. The costs and process of using this software implementation include: | ||
* Access to SaaS LIMS entails the payment of the initial LIMS license and account setup fees. These are one-time fees. As noted above, these charges are usually based on the number of concurrent users of the system (i.e., “seats”). A similar set of account-based use cases (Table 3) can be used to calculate various account scenarios and their license costs. | *Access to SaaS LIMS entails the payment of the initial LIMS license and account setup fees. These are one-time fees. As noted above, these charges are usually based on the number of concurrent users of the system (i.e., “seats”). A similar set of account-based use cases (Table 3) can be used to calculate various account scenarios and their license costs. | ||
* The next step is configuring the system to reflect your lab's requirements. This tends to be easier than an on-premises solution and would include setting up test descriptions (including test and product limits), personnel data, relevant initial customer data, report formats, access controls, and whatever else you define as needed to have a functioning system. This would not include instrument connections or connections to other systems. As noted earlier, while those can be and are done, they usually entail an additional cost. | *The next step is configuring the system to reflect your lab's requirements. This tends to be easier than an on-premises solution and would include setting up test descriptions (including test and product limits), personnel data, relevant initial customer data, report formats, access controls, and whatever else you define as needed to have a functioning system. This would not include instrument connections or connections to other systems. As noted earlier, while those can be and are done, they usually entail an additional cost. | ||
* Once the system is set up, a system test and acceptance process allows you to verify that the system is working according to your needs and specifications. This is part of the validation process but not the entire process. | *Once the system is set up, a system test and acceptance process allows you to verify that the system is working according to your needs and specifications. This is part of the validation process but not the entire process. | ||
* Recurring charges include the annual license renewal fee, usually a fraction of the initial license cost. This is usually simpler than with an on-premises solution. | *Recurring charges include the annual license renewal fee, usually a fraction of the initial license cost. This is usually simpler than with an on-premises solution. | ||
Table 4 provides a comparison of aspects of on-premises LIMS with cloud-based SaaS LIMS. | Table 4 provides a comparison of aspects of on-premises LIMS with cloud-based SaaS LIMS. | ||
Line 173: | Line 173: | ||
|} | |} | ||
In the end, from a purely cost-focused perspective, the cloud-based SaaS LIMS may make more sense for the modern lab. The previous chapter's Table 1 compared 1980s-era concessions, considerations, and justifications for maintaining an existing lab's operations vs. acquiring and deploying an on-premises LIMS. While those same aspects largely pertain to today's on-premises LIMS installation, most of the costs of hardware, installation, IT support, data storage, etc., go away with a cloud-based LIMS installation. Suddenly the economic justification for a LIMS looks rosier due to the reduced costs associated with having your LIMS securely hosted by another entity. The core cloud LIMS could be available on an annual basis for less than the cost of one person, easily justified through productivity improvements, improved data governance, and improved regulatory compliance. For example, the average annual salary for a government lab worker is | |||
$43,757 (2022 average)<ref name="FPLabWork23">{{cite web |url=https://www.federalpay.org/employees/occupations/laboratory-working |title=Occupation Index: Laboratory Working |work=FederalPay.org |date=2023 |accessdate=05 July 2023}}</ref>, which is very close to the base pay for a lab technician in Boston, Massachusetts (2023).<ref name="GDHowMuchLab23">{{cite web |url=https://www.glassdoor.com/Salaries/boston-lab-technician-salary-SRCH_IL.0,6_IM109_KO7,21.htm |title=How much does a Lab Technician make in Boston, MA? |work=Glassdoor |date=14 June 2023 |accessdate=05 July 2023}}</ref> | |||
The first year’s charges for a basic cloud-based LIMS with two concurrent users, including one-time and annual recurring charges, are less than that yearly salary value. The yearly recurring charge is less than a third of that technician's income. As you add on functions such as instrument interfacing (see the next subsection), both the initial and recurring costs will increase, and so will the productivity gains through faster, more efficient data entry and automation. Additionally, cloud-based LIMS largely eliminates the need for on-site server hardware, software installation, and IT support. (Note: It is not our position that people should be laid off to justify the added cost of a LIMS, but some future hiring may be avoided or deferred.) Yes, training still needs to be provided, as does support for instrument connections. However, the latter can be phased in on an as-needed basis. | |||
====2.2.2 Common and add-on costs==== | |||
It's important to note that regardless of the implementation approach, some costs are common to both on-premises and cloud-based SaaS systems. They include: | It's important to note that regardless of the implementation approach, some costs are common to both on-premises and cloud-based SaaS systems. They include: | ||
*''Networking'': Networking costs include wiring the lab and any recurring charges for support and the communications services themselves. These might come from a corporate budget or payments to an internet service provider. This should also include security protections against malware and intrusion. In the SaaS model, system security is provided by the vendor for the database system. An on-premises system is the lab’s or IT service's responsibility. Even with the SaaS model, your company has to provide network security to prevent the internet access services from becoming an entry point for hackers into your lab network. | |||
*''Training'': Everyone needs to be educated in the use of the LIMS. Some, like systems administrators, are educated to a higher degree than others. There is an initial charge for this when the system is introduced and a recurring charge as it evolves and new lab personnel come on board. Education for on-premises systems is more demanding since someone designated within the lab or IT services would function as the system administrator and handle all system modifications, including LIMS and underlying software upgrades. | |||
*''Printers'': The truly paperless lab may exist, but it is a rarity. People like paper; it's easy to work with and reassuring when you hold your results in your hand. | |||
*''System access'': Everyone needs system terminals that provide access to the LIMS and other systems in the lab. These include laptops, full computers, tablets, smartphones, and thin-client terminals. | |||
*''Project management'': Project management exists as a separate line item that comes in two forms, regardless of whether an on-premises or SaaS installation is used, though on-premises implementations would entail considerably more effort. One is the work the vendor does in implementing your requests and providing the initial support, while the other is the work that you do to generate user requirements, evaluating vendors, and managing the process from "We need a LIMS" to "its running and meets all specified requirements," and all the meetings and justifications that occur in between. That also includes having lab personnel educated on the use of the system. This is the basis for LIMS software process validation. Note that project management would be both an initial charge and a recurring cost. The latter accounts for updates to the user-controlled aspects of the system, such as adding new tests, clients, etc. (though software updates in SaaS environments are the vendor’s responsibility). | |||
Beyond those costs common to both implementation types, there are add-on capabilities. These can include technologies such as barcodes and instrument connections, as well as others. While this document doesn't intend to review all or a majority of the add-on capabilities, we will look at the two mentioned to illustrate a template for their justification. (Note, however, that barcode and instrument connection capabilities would not be realized with paper-based systems and are difficult to undertake with spreadsheet implementations.) | |||
Let's first address barcodes. Barcode technologies are an important alternative to manual data entry, greatly reducing the error rate.<ref name="Brady1Dv2D22">{{cite web |url=https://www.bradyid.com/applications/product-and-barcode-labeling/1d-vs-2d-barcodes |title=1D vs 2D Barcodes — What’s the Difference? |publisher=Brady Worldwide, Inc |date=04 October 2022 |accessdate=06 July 2023}}</ref> This is significant because the longer an error goes undetected, the greater the impact and cost repercussions: | |||
In the early 1990s, George Labovitz and Yu Sang Chang<ref>{{Cite book |last=Labovitz |first=George |last2=Chang |first2=Yu Sang |last3=Rosansky |first3=Victor |date=1992 |title=Making quality work: A leadership guide for the results-driven manager |publisher=Oliver Wight Ltd |place= |isbn=978-0-939246-54-0}}</ref> created the 1-10-100 rule, which states that data entry errors multiply costs exponentially according to the stage at which they are identified and corrected. If it costs you a dollar to fix a data entry error as soon as it's made, it will cost $10 at the next step of the process, perhaps when it is used as part of a calculation. If the error persists and is reported as part of an analytical sample report, it may cost $100 to fix, plus the embarrassment caused by the error. Those dollar figures are in 1992 valuations; in today’s dollars (2023), $100 then equals $214.43 today. | |||
The extent of error rate reduction will vary by barcode print quality, how clean the label is, the scanning equipment used, the user's proficiency, and the sample container's geometry. Plex Systems provides an error rate of 1 per 300 characters for manual entry, with it potentially being as good as 1 per 36 trillion characters for barcodes (this may be under ideal conditions).<ref name="PlexBarcoding">{{cite web |url=http://media.techtarget.com/Syndication/ENTERPRISE_APPS/Barcoding_WP.pdf |title=Barcoding 101 for Manufacturers: What You Need to Know to Get Started |publisher=Plex Systems |accessdate=06 July 2023}}</ref> ZIH Corp. states that the error rate of barcode data entry “has been widely estimated at one error per 3 million characters.”<ref name="ZIHBarcode16">{{cite web |url=https://www.zebra.com/content/dam/zebra_new_ia/en-us/solutions-verticals/vertical-solutions/healthcare/white-paper/barcode-labeling-lab-white%20paper-en-us.pdf |format=PDF |title=Barcode Labeling in the Lab — Closing the Loop of Patient Safety |publisher=ZIH Corp |date=March 2016 |accessdate=06 July 2023}}</ref> Peak Technologies states that data entered by barcodes are four times less likely to contain errors than that entered by hand.<ref name="KallenbachBarcodes20">{{cite web |url=https://www.dasco.com/resources/learning-center/2020/03/18/barcodes-in-the-lab/ |title=Barcodes in the Lab |author=Kallenbach, J. |publisher=Peak Technologies |date=18 March 2020 |accessdate=06 July 2023}}</ref> Finally, a CDC study reports that data entry errors are “nearly six times as likely from manual entry systems than from point-of-care test barcoding systems.”<ref name="LMBPWEffective12">{{cite web |url=https://www.cdc.gov/labbestpractices/pdfs/CDC_BarCodingSummary.pdf |format=PDF |title=Effective practices for reducing patient speciment and laboratory testing identification errors in diverse hospital settings |work=Laboratory Medicine Best Practices |author=Laboratory Medicine Best Practices Workgroup |publisher=Centers for Disease Control and Prevention |date=February 2012 |accessdate=06 July 2023}}</ref> There seems to be a disparity between studies that could be attributed to the difference between character counts and data entry measures where the data elements consist of multiple characters. | |||
As you might expect, any statement about error reduction will have significance throughout the LIMS justification. In addition to error reduction, barcodes are faster than manual entry, reducing it from 10 to 20 seconds per entry to possibly two or three. In addition, in some regulated environments, manual data entry requires verification by a second person, increasing the cost significantly. Barcodes can impact the sample login process, sample location and inventory, results entry, and almost any step in the sample processing sequence. | |||
Similar to Table 3, which provides a tabular template for examining user account costs, Table 5 gives you one possible way to tabulate a cost-benefit analysis of barcode technology. | |||
{| | |||
| style="vertical-align:top;" | | |||
{| class="wikitable" border="1" cellpadding="5" cellspacing="0" width="100%" | |||
|- | |||
| colspan="3" style="background-color:white; padding-left:10px; padding-right:10px;" |'''Table 5.''' A tabular cost-benefit analysis approach to barcode technology. | |||
|- | |||
! style="background-color:#e2e2e2; padding-left:10px; padding-right:10px;" colspan="2"|In-lab costs and benefits | |||
! style="background-color:#e2e2e2; padding-left:10px; padding-right:10px;" |Value | |||
|- | |||
| style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="4"|Cost of equipment | |||
| style="background-color:lightgrey; padding-left:10px; padding-right:10px;" colspan="2"| | |||
|- | |||
| style="background-color:white; padding-left:10px; padding-right:10px;"|Barcode readers | |||
| style="background-color:white; padding-left:10px; padding-right:10px;"| | |||
|- | |||
| style="background-color:white; padding-left:10px; padding-right:10px;"|Barcode printers | |||
| style="background-color:white; padding-left:10px; padding-right:10px;"| | |||
|- | |||
| style="background-color:white; padding-left:10px; padding-right:10px;"|Label stock for printing | |||
| style="background-color:white; padding-left:10px; padding-right:10px;"| | |||
|- | |||
| style="background-color:white; padding-left:10px; padding-right:10px;" rowspan="4"|Savings | |||
| style="background-color:lightgrey; padding-left:10px; padding-right:10px;" colspan="2"| | |||
|- | |||
| style="background-color:white; padding-left:10px; padding-right:10px;"|Time saved in data entry | |||
| style="background-color:white; padding-left:10px; padding-right:10px;"| | |||
|- | |||
| style="background-color:white; padding-left:10px; padding-right:10px;"|Reduced cost of error correction | |||
| style="background-color:white; padding-left:10px; padding-right:10px;"| | |||
|- | |||
| style="background-color:white; padding-left:10px; padding-right:10px;"|Savings due to faster sample processing | |||
| style="background-color:white; padding-left:10px; padding-right:10px;"| | |||
|- | |||
| style="background-color:white; padding-left:10px; padding-right:10px;" colspan="2"|Results (costs minus savings) | |||
| style="background-color:white; padding-left:10px; padding-right:10px;"| | |||
|- | |||
|} | |||
|} | |||
====2.2. | ====2.2.3 Factors that can offset costs==== | ||
===2.3 Practical considerations and justifications=== | ===2.3 Practical considerations and justifications=== |
Revision as of 00:13, 7 July 2023
This is sublevel9 of my sandbox, where I play with features and test MediaWiki code. If you wish to leave a comment for me, please see my discussion page instead. |
Sandbox begins below
2. Economic and practical justifications for a LIMS
As a lab manager or stakeholder in your organization, you've concluded that a laboratory information management system (LIMS) makes a lot of sense for solving some of the challenges your lab faces. But you're not the primary decision maker for LIMS acquisition and deployment. At some point, you will have to present your case (i.e., provide justification) for the LIMS to management, and the material included in this chapter will help you make that case. There are different approaches to the justification of LIMS in the laboratory; the methodology you adopt should be based on your understanding of the technology and the dynamics of your management structure. In some cases, presenting practical aspects of a core organizational system like LIMS will be sufficient, while in others a more rigorous analysis will be warranted.
This chapter will examine this justification process by first encouraging your organization to look at the factors that are closest to the lab's essential laboratory functions. From there, we can look at the more traditional economic and practical considerations, justifications, and benefits that can help you with your presentation to management. This allows us to lead into the following chapter, which will discuss the importance of management buy-in, as well as how to pitch the LIMS project to management and critical stakeholders.
2.1 Organizational justifications: Why is it important?
Before we get into the more classical aspects of justification, we need to look at those factors that are closest to your lab's essential operations. It's all well and good for us to broadly speak about the typical challenges, requirements, and considerations for labs of all types; some important deductions can be made by looking at the industry as a whole. However, no two laboratories are alike, and the challenges, requirements, and considerations for your laboratory may very well differ from the typical. This is why it's vital to not only use broad facts to justify LIMS acquisition but also integrate those facts within the context of your own lab. A series of important questions sets the stage for better explaining why the LIMS is necessary and beneficial to the lab. Questions that need to be asked include:
- Why is acquiring a LIMS important to meeting the goals of your lab?
- What problems does the LIMS solve that currently affect your lab?
- What operational, financial, and personnel improvements do you expect to see in your lab because of LIMS implementation?
- Why is this important to the larger organization, as well as those outside the lab?
No "one size fits all" justification template exists for a pivotal system like a LIMS. It can be the basis for restructuring your lab's operations and changing how it operates. Additionally, the justification for LIMS acquisition and deployment is—as the above questions emphasize—more than a dollars-and-cents consideration, although ultimately, it boils down to that. Management often thinks in terms of cost, and the further someone is in the corporate organizational chart from lab operations, the more financial issues become a driving factor in understanding the impact of a LIMS. They may need to be educated on the benefits of LIMS to lab operations, as well as the organization's bottom line. If lab budgets are tight, money may be a driving factor in that department. In other cases, it might be more straightforward, as reported by one oil company: “In our case, it pays for itself the day LIMS prevents one single day of refining gone to waste.”[1] Similar cases can be made for any regulated industry. Yes, it’s a money issue, but that is considerably removed from the lab's budget. The justification needs to be viewed from more than one perspective.
Let's now address each of those four questions.
2.1.1 Why is acquiring a LIMS important to meeting the goals of your lab?
Presumably, your organization has set out a series of business goals as part of a strategic planning process. Those developed goals should be purpose-driven, actionable, measurable, and forward-focused over a sufficiently long period of time to be fully indicative of the organization's long-term vision.[2] Having these goals on-hand are critical for any organization-specific justification of LIMS acquisition and deployment. These goals should, in theory, represent how upper management and key stakeholders view the organization's pathway to success. If you can match LIMS acquisition justification to those organizational goals, you'll have a better chance of getting through to management. If you can further make that goal-guided justification relevant and current to what's happening in the present, then all the better; "the goals that are timely and pressing are those that earn priority."[2]
Perhaps one of the organizational goals is for timely, accurate analytical results that are cleanly reported, with the belief that such service will earn repeat business in the future, which is a boon to the organization seeking consistency in its operational income. The lab manager who understands the benefits of a LIMS could then link a LIMS to fewer analytical errors and more timely error checking due to the automated nature of the system, as well as more rapid reporting and result distribution to clients. The lab manager could then tie these benefits to the original organizational goal, even providing theorized examples of improved return on investment (ROI) and how those savings could pay for the LIMS in relatively short order.
2.1.2 What problems does the LIMS solve that currently affect your lab?
If you're part of a mature, long-standing lab, there are likely to be some problems or challenges that have been identified as impacting your lab. Now is a good time to see how a LIMS can potentially aid with those problems and challenges. But new labs can also benefit; if your lab is new, the strategic planning process should have included risk analysis that successfully captures currently viewed and potential future risks to the business and achieving its mission-critical goals and priorities.[3] Drawing upon these risks helps you envision potential future problems you lab may encounter, as well as how a LIMS can help mitigate or prevent them.
An easy real-world example may be found with the challenges clinical labs encountered at the onset of the COVID-19 pandemic. Those with manual, paper-based workflows that still saw high-throughput activity buckled and cracked under the workloads that the pandemic pressed upon them.[4][5][6] Not only was keeping up with workloads on paper more difficult, but also tracking specimens and providing timely reporting to the Centers for Disease Control and Prevention (CDC) were especially challenging. Acquisition and implementation of a LIMS or laboratory information system (LIS) suddenly made more sense for its ability to better help manage increased test demand through improved test ordering methods, standardized order sets, timely shipping notifications, automated triaging, additional data mining, and smoother reporting both internally and to stakeholders (i.e., patients, governing bodies, policymakers).[6] Yes, there's a cost associated with the LIMS or LIS, but solving the paper-based challenge can make sense in the long-term for many such labs.
2.1.3 What operational, financial, and personnel improvements do you expect to see in your lab because of LIMS implementation?
You likely aren't pushing for the acquisition and deployment of a LIMS without some expectation of operational, financial, and/or personnel improvement occurring as a result. Of course, you can't go to management and wave around vague claims of improvement without any research and evidence to back up your assertions. This is where things get a bit tricky, potentially necessitating speculative examination of current and post-implementation workflow methods, ROI calculations, and surveys of lab personnel, to name only a few. It may also require an extensive demonstration of one or more potential LIMS by their respective vendors, including real-life examples of the lab's own analyses and workflows worked into the demonstrations. In the end, the lab manager and stakeholders will be looking to link LIMS implementation to expected improvements in quantifiable and qualifiable ways.
Perhaps a survey of laboratory personnel finds that 72 percent of them believe they'll save at least one hour of work a day for other important activities with the implementation of a LIMS. You then further examine the reasons for that belief and discover the current paper-based and spreadsheet methods involve a lot of manual order entry, additional footwork, waiting for spreadsheets to become available, and cleaning up of data entry errors. You then quantify the number of data entry errors and time spent on them, as well as the time spent entering orders, filing paperwork, waiting for resources, and performing quality checks on manually entered results. From there, you can readily imagine and demonstrate how a LIMS saves time and streamlines workflows, freeing up personnel for tasks that can't be automated and increasing productivity. That's something you can take to major stakeholders as part of your organization justification for a LIMS.
2.1.4 Why is this important to the larger organization, as well as those outside the lab?
The first three questions largely address the importance of a LIMS to the larger organization. But here the lab manager or key LIMS stakeholder has the opportunity to paint a broader picture of the importance of the LIMS in the scope of organizational goals, risks, challenges, and areas of improvement. "Importance implies a value judgement of the superior worth or influence of something," according to Merriam-Webster.[7] This value judgement can be expressed in many different ways, but you'll essentially be stating evidence that the LIMS is of "superior worth" to the current status quo. Many of the points made in the prior questions can be applied here, but you'll want to make the value judgement quantifiable, as well as qualifiable.
Also of interest with this question are stakeholders outside the lab. By extension, this generally implies that the value judgement of the LIMS relates also to the recipients of the analytical test data and information, whether they are patients, doctors, geologists, manufacturers, or regulatory bodies. Simply put, while the lab's and overall organization's impact are important, the lab and overall organization—through its business goals—ultimately serves its external stakeholders ordering or requiring reporting of laboratory analyses. The same value judgement and "superior worth" applied internally also must be applied to the data and information recipients. What do they gain by the lab shifting from paper-based methods to a LIMS? Are their results more timely and accurate? Are they more empowered through a results portal? Do they recommend you to other potential clients as a result? Some answers may be speculative, but similar to how lab personnel may get surveyed, one can imagine how conducting interviews with critical external stakeholders may reveal tangible evidence that shifting to a LIMS may prove to be an important move for the long-term future of the lab and its customers.
2.2 Economic considerations and justifications
Figure 2 shows the key cost-based considerations of a basic LIMS installation project, with details for both on-premises and cloud-based systems. Depending on the vendor, there are potential additional costs. For example, any additional instrument interfacing costs may depend on the vendor's approach, the number and type of instruments in your lab, and so on. The same applies to connections to other software systems (e.g., accounting, statistical data systems, and others). Instrument and software integration costs aside, Figure 2 also brings up:
- License, maintenance, and support costs: Whether one-time or recurring, there will be a license fee for the software. And it will require scheduled maintenance and support for when things go wrong.
- Acceptance and validation testing costs: Unless the vendor includes this in their implementation package, you may have to budget for the necessary tests to ensure the system is meeting the needs and requirements of your laboratory and its stakeholders.
- System and account configuration costs: You could try to have your soon-to-be system administrator manually add user accounts, configure roles, and make other configuration changes, or it may make sense to have the vendor do the work for you in advance, for a price.
- Data migration costs: If your lab already has electronic data of some sort, it may be viewed as mission-critical, with a need to modify and migrate it to the new LIMS in an acceptable format. Such data migration projects can be time-consuming and costly, depending on the complexity and "dirtiness" of the data being imported.
- Networking costs: Whether installed locally or in the cloud, some sort of networking is required (unless your a tiny lab with a single computer, running software locally). The client instances of the LIMS will need to electronically communicate with the server instance, or the web-based access screen will need to electronically communicate with the vendor hosting the LIMS in the cloud. This means local networking costs and/or a highly-dependable internet connection.
- Supporting equipment and hardware costs: From barcode readers and label printers to system terminals and document printers, the implementation of your LIMS may require supporting equipment that, in tandem with the LIMS, will improve workflows and expedite laboratory tasks. The LIMS introduces improved sample and specimen tracking using barcode labels and scanners, which benefits the lab, but is useless without the additional equipment. If the LIMS is to be hosted on-premises, IT hardware will also likely need to be acquired.
- Training costs: Some vendors may include a set amount of training with their implementation package, while others may not. Your lab is likely going to want to hit the ground running with the new LIMS, which means well-trained personnel, with full documentation and recorded training videos. As new personnel are on-boarded, your lab may not want to be the one responsible for training, opting to ask the vendor to run the employee through training.
- Project management costs: Acquiring and implementing a LIMS is a project, full stop. The costs associated with project management may be handled internally, through normal hourly- or salary-based payment of personnel and managers for their organizational duties. However, some labs may not have the internal resources and experience to field such a project, requiring either a consultancy or even hiring the LIMS vendor to work with a lab stakeholder to oversee the project.
|
As you'll see in the following subsections, the method of accounting for these and other costs associated with LIMS implementation, and their justification, will be similar to that of acquiring supporting laboratory equipment: what are the costs, and what are the benefits, both tangible and intangible?
2.2.1 On-premises vs. cloud LIMS
The economics behind the on-premises LIMS are generally different from those of the cloud-based LIMS. An on-premises system ultimately sees the system administrator taking responsibility for most or all aspects of the system. This includes:
- Hardware selection: This includes the computer and associated components (e.g., printer, power backup, software backup, etc.). Additionally, the admin will need to find a place to put the computer and make arrangements with IT support for their services.
- LIMS installation: This includes installation of the operating system, LIMS software, and any underlying components (e.g., database software, support tools, etc.). This may be done alone or with the vendor's and IT services' assistance.
- LIMS validation and testing: The admin will be responsible for full system validation and testing. This significantly increases the workload of project management and support.
The LIMS-specific software costs and process of using this type of LIMS implementation include the following:
- Payment of the initial LIMS license and account setup fees is a one-time charge. These charges are usually based on the number of concurrent users of the system (sometimes referred to as “seats”). If you have six people in your lab, each can have their own login (meeting regulatory requirements), but only the specified number of concurrent users can access the software simultaneously. For example, with six accounts, and a license for two seats, two people can be working on the LIMS simultaneously. It may help to review several use cases internally, as well as their associated costs, to get a sense for how many accounts you need and can afford. Table 3 provides an example of a tabular view one might take with calculating license costs for several use cases.
|
- Once the software has been installed and passes the necessary qualification testing, the next step is to configure the system to reflect your lab's requirements. This is more demanding than a cloud-based software as a service (SaaS) implementation since the project manager (PM) is responsible for all aspects of the work instead of being divided between the PM and the vendor; the vendor may provide guidance for some of this work based on their experience. This would include setting up test descriptions (including test and product limits), personnel data, relevant initial customer data, report formats, access controls, etc., whatever you define as needed to have a functioning system. This would not include instrument connections or connections to other systems. While those can be and are done, they usually entail an additional cost. Issues like that, and the details of their inclusion, are beyond the scope of this document, but you should be able to include them easily. The costs for instrument connections can vary widely depending on the devices' nature, use, and vendor’s support.
- Once the system is set up, a system test and acceptance process begins as a part of a complete system acceptance/validation program. This compares the contents of the user requirement document to the functioning systems. This allows you to verify that the system works according to your needs and specifications. This is part of the validation process but not the entire process.
- Recurring charges include the annual license renewal fee, usually a fraction of the initial license cost. In addition, there would be hardware support costs, software maintenance fees (e.g., underlying components, including the operating system), backup systems annual fees (e.g., power, hardware, software), and IT support costs. Support can involve software and hardware vendors, facilities management (e.g., backup power), and IT support.
Cloud-based SaaS LIMS, on the other hand, are hosted on the vendor's server and is ready to be accessed and used once it is initialized with your lab's specific details. No local software installation is needed. The costs and process of using this software implementation include:
- Access to SaaS LIMS entails the payment of the initial LIMS license and account setup fees. These are one-time fees. As noted above, these charges are usually based on the number of concurrent users of the system (i.e., “seats”). A similar set of account-based use cases (Table 3) can be used to calculate various account scenarios and their license costs.
- The next step is configuring the system to reflect your lab's requirements. This tends to be easier than an on-premises solution and would include setting up test descriptions (including test and product limits), personnel data, relevant initial customer data, report formats, access controls, and whatever else you define as needed to have a functioning system. This would not include instrument connections or connections to other systems. As noted earlier, while those can be and are done, they usually entail an additional cost.
- Once the system is set up, a system test and acceptance process allows you to verify that the system is working according to your needs and specifications. This is part of the validation process but not the entire process.
- Recurring charges include the annual license renewal fee, usually a fraction of the initial license cost. This is usually simpler than with an on-premises solution.
Table 4 provides a comparison of aspects of on-premises LIMS with cloud-based SaaS LIMS.
|
In the end, from a purely cost-focused perspective, the cloud-based SaaS LIMS may make more sense for the modern lab. The previous chapter's Table 1 compared 1980s-era concessions, considerations, and justifications for maintaining an existing lab's operations vs. acquiring and deploying an on-premises LIMS. While those same aspects largely pertain to today's on-premises LIMS installation, most of the costs of hardware, installation, IT support, data storage, etc., go away with a cloud-based LIMS installation. Suddenly the economic justification for a LIMS looks rosier due to the reduced costs associated with having your LIMS securely hosted by another entity. The core cloud LIMS could be available on an annual basis for less than the cost of one person, easily justified through productivity improvements, improved data governance, and improved regulatory compliance. For example, the average annual salary for a government lab worker is $43,757 (2022 average)[8], which is very close to the base pay for a lab technician in Boston, Massachusetts (2023).[9]
The first year’s charges for a basic cloud-based LIMS with two concurrent users, including one-time and annual recurring charges, are less than that yearly salary value. The yearly recurring charge is less than a third of that technician's income. As you add on functions such as instrument interfacing (see the next subsection), both the initial and recurring costs will increase, and so will the productivity gains through faster, more efficient data entry and automation. Additionally, cloud-based LIMS largely eliminates the need for on-site server hardware, software installation, and IT support. (Note: It is not our position that people should be laid off to justify the added cost of a LIMS, but some future hiring may be avoided or deferred.) Yes, training still needs to be provided, as does support for instrument connections. However, the latter can be phased in on an as-needed basis.
2.2.2 Common and add-on costs
It's important to note that regardless of the implementation approach, some costs are common to both on-premises and cloud-based SaaS systems. They include:
- Networking: Networking costs include wiring the lab and any recurring charges for support and the communications services themselves. These might come from a corporate budget or payments to an internet service provider. This should also include security protections against malware and intrusion. In the SaaS model, system security is provided by the vendor for the database system. An on-premises system is the lab’s or IT service's responsibility. Even with the SaaS model, your company has to provide network security to prevent the internet access services from becoming an entry point for hackers into your lab network.
- Training: Everyone needs to be educated in the use of the LIMS. Some, like systems administrators, are educated to a higher degree than others. There is an initial charge for this when the system is introduced and a recurring charge as it evolves and new lab personnel come on board. Education for on-premises systems is more demanding since someone designated within the lab or IT services would function as the system administrator and handle all system modifications, including LIMS and underlying software upgrades.
- Printers: The truly paperless lab may exist, but it is a rarity. People like paper; it's easy to work with and reassuring when you hold your results in your hand.
- System access: Everyone needs system terminals that provide access to the LIMS and other systems in the lab. These include laptops, full computers, tablets, smartphones, and thin-client terminals.
- Project management: Project management exists as a separate line item that comes in two forms, regardless of whether an on-premises or SaaS installation is used, though on-premises implementations would entail considerably more effort. One is the work the vendor does in implementing your requests and providing the initial support, while the other is the work that you do to generate user requirements, evaluating vendors, and managing the process from "We need a LIMS" to "its running and meets all specified requirements," and all the meetings and justifications that occur in between. That also includes having lab personnel educated on the use of the system. This is the basis for LIMS software process validation. Note that project management would be both an initial charge and a recurring cost. The latter accounts for updates to the user-controlled aspects of the system, such as adding new tests, clients, etc. (though software updates in SaaS environments are the vendor’s responsibility).
Beyond those costs common to both implementation types, there are add-on capabilities. These can include technologies such as barcodes and instrument connections, as well as others. While this document doesn't intend to review all or a majority of the add-on capabilities, we will look at the two mentioned to illustrate a template for their justification. (Note, however, that barcode and instrument connection capabilities would not be realized with paper-based systems and are difficult to undertake with spreadsheet implementations.)
Let's first address barcodes. Barcode technologies are an important alternative to manual data entry, greatly reducing the error rate.[10] This is significant because the longer an error goes undetected, the greater the impact and cost repercussions:
In the early 1990s, George Labovitz and Yu Sang Chang[11] created the 1-10-100 rule, which states that data entry errors multiply costs exponentially according to the stage at which they are identified and corrected. If it costs you a dollar to fix a data entry error as soon as it's made, it will cost $10 at the next step of the process, perhaps when it is used as part of a calculation. If the error persists and is reported as part of an analytical sample report, it may cost $100 to fix, plus the embarrassment caused by the error. Those dollar figures are in 1992 valuations; in today’s dollars (2023), $100 then equals $214.43 today.
The extent of error rate reduction will vary by barcode print quality, how clean the label is, the scanning equipment used, the user's proficiency, and the sample container's geometry. Plex Systems provides an error rate of 1 per 300 characters for manual entry, with it potentially being as good as 1 per 36 trillion characters for barcodes (this may be under ideal conditions).[12] ZIH Corp. states that the error rate of barcode data entry “has been widely estimated at one error per 3 million characters.”[13] Peak Technologies states that data entered by barcodes are four times less likely to contain errors than that entered by hand.[14] Finally, a CDC study reports that data entry errors are “nearly six times as likely from manual entry systems than from point-of-care test barcoding systems.”[15] There seems to be a disparity between studies that could be attributed to the difference between character counts and data entry measures where the data elements consist of multiple characters.
As you might expect, any statement about error reduction will have significance throughout the LIMS justification. In addition to error reduction, barcodes are faster than manual entry, reducing it from 10 to 20 seconds per entry to possibly two or three. In addition, in some regulated environments, manual data entry requires verification by a second person, increasing the cost significantly. Barcodes can impact the sample login process, sample location and inventory, results entry, and almost any step in the sample processing sequence.
Similar to Table 3, which provides a tabular template for examining user account costs, Table 5 gives you one possible way to tabulate a cost-benefit analysis of barcode technology.
|
2.2.3 Factors that can offset costs
2.3 Practical considerations and justifications
2.3.1 Tangible benefits
2.3.2 Intangible benefits
References
- ↑ Segalstad, S. (18 May 2015). "Cost of LIMS: True Pricing includes more than Purchase, Implementation and Annual Licensing". R&D World. https://www.rdworldonline.com/cost-of-lims-true-pricing-includes-more-than-purchase-implementation-and-annual-licensing/. Retrieved 05 July 2023.
- ↑ 2.0 2.1 Cote, C. (29 October 2020). "How to Set Strategic Planning Goals". Business Insights Blog. Harvard Business School Online. https://online.hbs.edu/blog/post/strategic-planning-goals. Retrieved 05 July 2023.
- ↑ "Strategic Planning Essentials" (PDF). Gartner, Inc. 2023. https://emtemp.gcom.cloud/ngw/globalassets/en/insights/strategic-planning/2023/documents/strategic-planning-ebook-2023-risk.pdf. Retrieved 05 July 2023.
- ↑ "A Vision for Action in Digital Health, 2020-2024" (PDF). USAID. May 2022. https://www.usaid.gov/sites/default/files/2022-05/USAID-A-Digital-Health-Vision-for-Action-v10.28_FINAL_508.pdf. Retrieved 05 July 2023.
- ↑ Barnes, C. (June 2022). "Going paperless: The world has changed" (PDF). Pathology Focus (Australian Clinicalabs) (18). https://www.clinicallabs.com.au/media/4165/pathology-focus-newsletter-june-2022-vic-qld-aclmar-news-nat-03898-digital.pdf. Retrieved 05 July 2023.
- ↑ 6.0 6.1 Weemaes, Matthias; Martens, Steven; Cuypers, Lize; Van Elslande, Jan; Hoet, Katrien; Welkenhuysen, Joris; Goossens, Ria; Wouters, Stijn et al. (1 August 2020). "Laboratory information system requirements to manage the COVID-19 pandemic: A report from the Belgian national reference testing center" (in en). Journal of the American Medical Informatics Association 27 (8): 1293–1299. doi:10.1093/jamia/ocaa081. ISSN 1527-974X. PMC PMC7197526. PMID 32348469. https://academic.oup.com/jamia/article/27/8/1293/5827002.
- ↑ "importance". Merriam-Webster Dictionary. Merriam-Webster. https://www.merriam-webster.com/dictionary/importance. Retrieved 05 July 2023.
- ↑ "Occupation Index: Laboratory Working". FederalPay.org. 2023. https://www.federalpay.org/employees/occupations/laboratory-working. Retrieved 05 July 2023.
- ↑ "How much does a Lab Technician make in Boston, MA?". Glassdoor. 14 June 2023. https://www.glassdoor.com/Salaries/boston-lab-technician-salary-SRCH_IL.0,6_IM109_KO7,21.htm. Retrieved 05 July 2023.
- ↑ "1D vs 2D Barcodes — What’s the Difference?". Brady Worldwide, Inc. 4 October 2022. https://www.bradyid.com/applications/product-and-barcode-labeling/1d-vs-2d-barcodes. Retrieved 06 July 2023.
- ↑ Labovitz, George; Chang, Yu Sang; Rosansky, Victor (1992). Making quality work: A leadership guide for the results-driven manager. Oliver Wight Ltd. ISBN 978-0-939246-54-0.
- ↑ "Barcoding 101 for Manufacturers: What You Need to Know to Get Started". Plex Systems. http://media.techtarget.com/Syndication/ENTERPRISE_APPS/Barcoding_WP.pdf. Retrieved 06 July 2023.
- ↑ "Barcode Labeling in the Lab — Closing the Loop of Patient Safety" (PDF). ZIH Corp. March 2016. https://www.zebra.com/content/dam/zebra_new_ia/en-us/solutions-verticals/vertical-solutions/healthcare/white-paper/barcode-labeling-lab-white%20paper-en-us.pdf. Retrieved 06 July 2023.
- ↑ Kallenbach, J. (18 March 2020). "Barcodes in the Lab". Peak Technologies. https://www.dasco.com/resources/learning-center/2020/03/18/barcodes-in-the-lab/. Retrieved 06 July 2023.
- ↑ Laboratory Medicine Best Practices Workgroup (February 2012). "Effective practices for reducing patient speciment and laboratory testing identification errors in diverse hospital settings" (PDF). Laboratory Medicine Best Practices. Centers for Disease Control and Prevention. https://www.cdc.gov/labbestpractices/pdfs/CDC_BarCodingSummary.pdf. Retrieved 06 July 2023.
Citation information for this chapter
Chapter: 2. Economic and practical justifications for a LIMS
Title: Justifying LIMS Acquisition and Deployment within Your Organization
Edition: First Edition
Author for citation: Joe Liscouski, Shawn E. Douglas
License for content: Creative Commons Attribution-ShareAlike 4.0 International
Publication date: