Difference between revisions of "User:Shawndouglas/sandbox/sublevel5"
Shawndouglas (talk | contribs) |
Shawndouglas (talk | contribs) |
||
Line 19: | Line 19: | ||
==Examples of LIMS alternatives== | ==Examples of LIMS alternatives== | ||
A [[laboratory information management system]] (LIMS) is a modern solution to the increasingly demanding [[workflow]] needs of most [[Laboratory|laboratories]], particularly those performing activities in regulated industries. However, laboratory work wasn't always conducted with the help of such a software system, and a LIMS isn't always the answer for a lab looking to manage its workflow and operations. Whether it's the simplicity of the lab's operations or the perceived costs of acquiring, maintaining, and updating a LIMS (or even heavy stakeholder resistance to updating old familiar processes to more modern ones), other alternatives still exist for laboratories, including paper-based systems, spreadsheet software, [[database]] software, or [[enterprise resource planning]] (ERP) software. This section will briefly discuss those options, while the following section will address their potential deficiencies. | A [[laboratory information management system]] (LIMS) is a modern solution to the increasingly demanding [[workflow]] needs of most [[Laboratory|laboratories]], particularly those performing activities in regulated industries. However, laboratory work wasn't always conducted with the help of such a software system, and a LIMS isn't always the answer for a lab looking to better manage its workflow and operations. Whether it's the simplicity of the lab's operations or the perceived costs of acquiring, maintaining, and updating a LIMS (or even heavy stakeholder resistance to updating old familiar processes to more modern ones), other alternatives still exist for laboratories, including paper-based systems, spreadsheet software, [[database]] software, or [[enterprise resource planning]] (ERP) software. This section will briefly discuss those options, while the following section will address their potential deficiencies. | ||
===Paper-based systems=== | ===Paper-based systems=== | ||
Line 25: | Line 25: | ||
===Spreadsheets=== | ===Spreadsheets=== | ||
As computing technology evolved and became more affordable, software makers had even more incentive to develop relevant and approachable software solutions to solve businesses' workflow challenges. Among these software solutions was the spreadsheet. Derived in concept from the paper-based ledgers accountants and traders would use, the electronic spreadsheet suddenly allowed businesses to perform calculations automatically, saving users time.<ref name="MeikleTheHist">{{cite web |url=https://blog.sheetgo.com/spreadsheets-tips/history-of-spreadsheets/ |title=The history of spreadsheets |author=Meikle, H. |work=Sheetgo Blog |accessdate=15 December 2023}}</ref> Laboratories picked up on this electronic, ledger-based approach to documenting experimental results and making routine analytical calculations. However, as labs of all types have fallen under greater scrutiny from regulators, the spreadsheet method of documentation and calculation of analytical results shows inefficiencies and inadequacies, including difficulty in preventing changes to fields and maintaining an accurate representation of the who, what, when, and where of recorded values. | As computing technology evolved and became more affordable, software makers had even more incentive to develop relevant and approachable software solutions to solve businesses' workflow challenges. Among these software solutions was the spreadsheet. Derived in concept from the paper-based ledgers accountants and traders would use, the electronic spreadsheet suddenly allowed businesses to perform calculations automatically, saving users time.<ref name="MeikleTheHist">{{cite web |url=https://blog.sheetgo.com/spreadsheets-tips/history-of-spreadsheets/ |title=The history of spreadsheets |author=Meikle, H. |work=Sheetgo Blog |accessdate=15 December 2023}}</ref> Laboratories picked up on this electronic, ledger-based approach to documenting experimental results and making routine analytical calculations. However, as labs of all types have fallen under greater scrutiny from regulators, the electronic spreadsheet method of documentation and calculation of analytical results shows inefficiencies and inadequacies, including difficulty in preventing changes to fields and maintaining an accurate representation of the who, what, when, and where of recorded values. | ||
===Databases=== | ===Databases=== | ||
Databases also came into popularity with the advent of computing technology. Tabular and relational representation of data points, with the ability to assign labels to those data points, became useful for the electronic storage and retrieval of all sorts of data.<ref name="FortuneABrief20">{{cite web |url=https://learn.saylor.org/mod/page/view.php?id=21059 |title=A Brief History of Databases |work=CS403: Introduction to Modern Database Systems |author=Fortune, S. |publisher=Saylor Academy |date=17 December 2020 |accessdate=15 December 2023}}</ref> Like the spreadsheet, it's not surprising that some laboratories latched on to the idea of keeping track of experimental, analytical, and [[quality control]] data in a database. However, these are best used for structured data, and as electronic types of data and information have evolved into more sophisticated forms such as images, audio, and other unstructured formats, the database has | Databases also came into popularity with the advent of computing technology. Tabular and relational representation of data points, with the ability to assign labels to those data points, became useful for the electronic storage and retrieval of all sorts of data.<ref name="FortuneABrief20">{{cite web |url=https://learn.saylor.org/mod/page/view.php?id=21059 |title=A Brief History of Databases |work=CS403: Introduction to Modern Database Systems |author=Fortune, S. |publisher=Saylor Academy |date=17 December 2020 |accessdate=15 December 2023}}</ref> Like the spreadsheet, it's not surprising that some laboratories latched on to the idea of keeping track of experimental, analytical, and [[quality control]] data in a database. However, these are best used for structured data, and as electronic types of data and information have evolved into more sophisticated forms such as images, audio, and other unstructured formats, the traditional database has shown its weaknesses. | ||
===ERPs=== | ===ERPs=== | ||
Compared to spreadsheets and databases, the ERP is a bit more modern software solution, specifically designed to help streamline and unify business processes across an entire organization. These systems were originally designed for larger enterprises such as banks, manufacturers, and insurance companies, helping them manage finances, assets, inventory and supply chain, training, and other business aspects. However, even today most don't have the functionality necessary to handle more laboratory-specific activities, stretching the ERP's functionality to the limit. (There are examples of savvy software vendors who've built laboratory-specific modules for existing ERPs like open-source [[Odoo]]<ref name="LSOdooLIMS">{{cite web |url=https://www.logicasoft.eu/en_US/lims |title=Odoo LIMS |publisher=LogicaSoft SPRL |accessdate=15 December 2023}}</ref>, but these appear to be few and far between.) | Compared to spreadsheets and databases, the ERP is a bit more modern software solution, specifically designed to help streamline and unify business processes across an entire organization. These systems were originally designed for larger enterprises such as banks, manufacturers, and insurance companies, helping them manage finances, assets, inventory and supply chain, training, and other business aspects. However, even today most don't have the functionality necessary to handle more laboratory-specific activities, stretching the ERP's functionality to the limit. (There are examples of savvy software vendors who've built laboratory-specific modules for existing ERPs like [[Open-source software|open-source]] [[Odoo]]<ref name="LSOdooLIMS">{{cite web |url=https://www.logicasoft.eu/en_US/lims |title=Odoo LIMS |publisher=LogicaSoft SPRL |accessdate=15 December 2023}}</ref>, but these appear to be few and far between.) | ||
==Deficiencies in most LIMS alternatives== | ==Deficiencies in most LIMS alternatives== | ||
To be sure, a LIMS is an investment for any sized laboratory, whether it's almost exclusively monetary (with some other organization doing a majority of the heavy lifting, as with a [[Cloud computing|cloud-based]] solution) or some combination of monetary and in-house resource expenditure (as with a self-hosted solution located on-premises, whether that solution is a commercial proprietary offering or an | To be sure, a LIMS is an investment for any sized laboratory, whether it's almost exclusively monetary (with some other organization doing a majority of the heavy lifting, as with a [[Cloud computing|cloud-based]] solution) or some combination of monetary and in-house resource expenditure (as with a self-hosted solution located on-premises, whether that solution is a commercial proprietary offering or an open-source offering). Even an open-source LIMS still requires the lab to lean on an employee or third-party consultant to set up, configure, and maintain the software (or even modify the source code), as well as maintain the local IT infrastructure to support it. The open-source route may make sense for small, single labs with a couple of instruments, but the lack of regulatory-driven functionality like an [[audit trail]] in all but a few open-source LIMS (e.g., [[SENAITE]]<ref name="SENAITEFeats">{{cite web |url=https://www.senaite.com/features |title=SENAITE - Features |publisher=SENAITE Foundation |accessdate=13 December 2023}}</ref>) may significantly restrict the available options to such labs. | ||
This brings up the point of what a lab typically sacrifices with LIMS alternatives such as paper notebooks, spreadsheets, databases, and ERPs. These alternatives rarely address regulatory and/or internal need for<ref name="LiscouskiImprov22">{{cite web |url=https://www.lablynx.com/wp-content/uploads/2023/03/Improving-Lab-Systems-From-Paper-to-Spreadsheets-to-LIMS.pdf |format=PDF |title=Improving Lab Systems: From Paper to Spreadsheets to LIMS |author=Liscouski, J. |publisher=LabLynx, Inc. |date=April 2022 |accessdate=13 December 2023}}</ref><ref name="LiscouskiASci23">{{cite web |url=https://www.limswiki.org/index.php/LII:A_Science_Student%27s_Guide_to_Laboratory_Informatics |title=LII:A Science Student's Guide to Laboratory Informatics |author=Liscouski, J. |editor=Douglas, S.E. |work=LIMSwiki.org |date=November 2023 |accessdate=13 December 2023}}</ref><ref>{{Cite book |last=Ferrero, M.S. |date=2007 |editor-last=Weinberg |editor-first=Sandy |title=Good laboratory practice regulations |url=https://books.google.com/books?id=JR5i0Nz5UWEC&pg=PA233 |chapter=Chapter 8: GLP Documentation |series=Drugs and the pharmaceutical sciences |edition=4th ed |publisher=Informa Healthcare |place=New York |pages=223–240 |isbn=9780849375842}}</ref><ref>{{Cite journal |last=AlTarawneh |first=Ghada |last2=Thorne |first2=Simon |date=2017 |title=A Pilot Study Exploring Spreadsheet Risk in Scientific Research |url=https://arxiv.org/abs/1703.09785 |journal=arXiv |doi=10.48550/ARXIV.1703.09785}}</ref><ref name="McDowallAreSpr20">{{cite journal |url=https://www.chromatographyonline.com/view/are-spreadsheets-a-fast-track-to-regulatory-non-compliance |title=Are Spreadsheets a Fast Track to Regulatory Non-Compliance? |author=McDowall, R.D. |journal=LCGC Europe |volume=33 |issue=9 |pages=468–76 |year=2020 |accessdate=13 December 2023}}</ref><ref>{{Cite journal |last=Ma |first=Ming-Wei |last2=Gao |first2=Xian-Shu |last3=Zhang |first3=Ze-Yu |last4=Shang |first4=Shi-Yu |last5=Jin |first5=Ling |last6=Liu |first6=Pei-Lin |last7=Lv |first7=Feng |last8=Ni |first8=Wei |last9=Han |first9=Yu-Chen |last10=Zong |first10=Hui |date=2023-11-06 |title=Extracting laboratory test information from paper-based reports |url=https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-023-02346-6 |journal=BMC Medical Informatics and Decision Making |language=en |volume=23 |issue=1 |pages=251 |doi=10.1186/s12911-023-02346-6 |issn=1472-6947 |pmc=PMC10629084 |pmid=37932733}}</ref><ref>{{Citation |last=McDowall. R.D. |date= |year=2018 |title=Chapter 13. Get Rid of Paper: Why Electronic Processes are Better for Data Integrity |url=http://ebook.rsc.org/?DOI=10.1039/9781788013277-00281 |work=Data Integrity and Data Governance |language=en |publisher=Royal Society of Chemistry |place=Cambridge |pages=281–304 |doi=10.1039/9781788013277-00281 |isbn=978-1-78801-281-2 |accessdate=}}</ref>: | This brings up the point of what a lab typically sacrifices with LIMS alternatives such as paper notebooks, spreadsheets, databases, and ERPs. These alternatives rarely address regulatory and/or internal need for<ref name="LiscouskiImprov22">{{cite web |url=https://www.lablynx.com/wp-content/uploads/2023/03/Improving-Lab-Systems-From-Paper-to-Spreadsheets-to-LIMS.pdf |format=PDF |title=Improving Lab Systems: From Paper to Spreadsheets to LIMS |author=Liscouski, J. |publisher=LabLynx, Inc. |date=April 2022 |accessdate=13 December 2023}}</ref><ref name="LiscouskiASci23">{{cite web |url=https://www.limswiki.org/index.php/LII:A_Science_Student%27s_Guide_to_Laboratory_Informatics |title=LII:A Science Student's Guide to Laboratory Informatics |author=Liscouski, J. |editor=Douglas, S.E. |work=LIMSwiki.org |date=November 2023 |accessdate=13 December 2023}}</ref><ref>{{Cite book |last=Ferrero, M.S. |date=2007 |editor-last=Weinberg |editor-first=Sandy |title=Good laboratory practice regulations |url=https://books.google.com/books?id=JR5i0Nz5UWEC&pg=PA233 |chapter=Chapter 8: GLP Documentation |series=Drugs and the pharmaceutical sciences |edition=4th ed |publisher=Informa Healthcare |place=New York |pages=223–240 |isbn=9780849375842}}</ref><ref>{{Cite journal |last=AlTarawneh |first=Ghada |last2=Thorne |first2=Simon |date=2017 |title=A Pilot Study Exploring Spreadsheet Risk in Scientific Research |url=https://arxiv.org/abs/1703.09785 |journal=arXiv |doi=10.48550/ARXIV.1703.09785}}</ref><ref name="McDowallAreSpr20">{{cite journal |url=https://www.chromatographyonline.com/view/are-spreadsheets-a-fast-track-to-regulatory-non-compliance |title=Are Spreadsheets a Fast Track to Regulatory Non-Compliance? |author=McDowall, R.D. |journal=LCGC Europe |volume=33 |issue=9 |pages=468–76 |year=2020 |accessdate=13 December 2023}}</ref><ref>{{Cite journal |last=Ma |first=Ming-Wei |last2=Gao |first2=Xian-Shu |last3=Zhang |first3=Ze-Yu |last4=Shang |first4=Shi-Yu |last5=Jin |first5=Ling |last6=Liu |first6=Pei-Lin |last7=Lv |first7=Feng |last8=Ni |first8=Wei |last9=Han |first9=Yu-Chen |last10=Zong |first10=Hui |date=2023-11-06 |title=Extracting laboratory test information from paper-based reports |url=https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-023-02346-6 |journal=BMC Medical Informatics and Decision Making |language=en |volume=23 |issue=1 |pages=251 |doi=10.1186/s12911-023-02346-6 |issn=1472-6947 |pmc=PMC10629084 |pmid=37932733}}</ref><ref>{{Citation |last=McDowall. R.D. |date= |year=2018 |title=Chapter 13. Get Rid of Paper: Why Electronic Processes are Better for Data Integrity |url=http://ebook.rsc.org/?DOI=10.1039/9781788013277-00281 |work=Data Integrity and Data Governance |language=en |publisher=Royal Society of Chemistry |place=Cambridge |pages=281–304 |doi=10.1039/9781788013277-00281 |isbn=978-1-78801-281-2 |accessdate=}}</ref>: | ||
*ensuring analytical results haven't been maliciously or accidentally modified (such as with audit trails that clearly and properly maintain the metadata surrounding an inputted value, as well as any changes made to it); | *ensuring analytical results haven't been maliciously or accidentally modified (such as with audit trails that clearly and properly maintain the [[metadata]] surrounding an inputted value, as well as any changes made to it); | ||
*clearly and accurately | *clearly and accurately documenting a wide variety of data and metadata about a given sample or analysis; | ||
*ensuring software tools like spreadsheets are validated; | *ensuring software tools like spreadsheets are validated; | ||
*ensuring data is contemporaneous (i.e., "current") and not fabricated post-analysis; | *ensuring data is contemporaneous (i.e., "current") and not fabricated post-analysis; | ||
Line 52: | Line 52: | ||
*supporting later conversion of paper-based data and information into structured, readable, and importable electronic formats. | *supporting later conversion of paper-based data and information into structured, readable, and importable electronic formats. | ||
This is not to say that paper-based laboratory notebooks, spreadsheets, databases, and ERPs can't work for small, lightly-regulated laboratories. However, as more laboratory activities across all industries gain more regulatory oversight, and as clientele of said labs increasingly demand more timely, accurate, and defensible analytical results, today's laboratories are under pressure to move beyond little print-out slips from instruments, paper notebooks, controlled worksheets, and non-validated software tools that aren't purpose-built for labs. | This is not to say that paper-based laboratory notebooks, spreadsheets, databases, and ERPs can't work for small, lightly-regulated laboratories. However, as more laboratory activities across all industries gain more regulatory oversight, and as clientele of said labs increasingly demand more timely, accurate, and defensible analytical results, today's laboratories are under pressure to move beyond little print-out slips from instruments, paper notebooks, controlled worksheets, and non-validated software tools that aren't purpose-built for labs. That some LIMS vendors have recognized that not every lab needs a megalithic software solution—in turn offering slimmed-down, more affordable LIMS solutions—is even more encouraging for small labs that want to take the next step towards improved workflows and greater data integrity. | ||
==Conclusion== | ==Conclusion== |
Revision as of 15:39, 16 December 2023
This is sublevel5 of my sandbox, where I play with features and test MediaWiki code. If you wish to leave a comment for me, please see my discussion page instead. |
Sandbox begins below
[[File:|right|300px]] Title: What are the alternatives to a laboratory information management system (LIMS)?
Author for citation: Shawn E. Douglas
License for content: Creative Commons Attribution-ShareAlike 4.0 International
Publication date: December 2023
Introduction
Examples of LIMS alternatives
A laboratory information management system (LIMS) is a modern solution to the increasingly demanding workflow needs of most laboratories, particularly those performing activities in regulated industries. However, laboratory work wasn't always conducted with the help of such a software system, and a LIMS isn't always the answer for a lab looking to better manage its workflow and operations. Whether it's the simplicity of the lab's operations or the perceived costs of acquiring, maintaining, and updating a LIMS (or even heavy stakeholder resistance to updating old familiar processes to more modern ones), other alternatives still exist for laboratories, including paper-based systems, spreadsheet software, database software, or enterprise resource planning (ERP) software. This section will briefly discuss those options, while the following section will address their potential deficiencies.
Paper-based systems
Before software-based systems, labs used laboratory notebooks, notepads, Xeroxed report templates, and instrument printouts (granted, a step up from recording from pure observation) to document analytical and research processes. Even in 2023, we still find labs that stick to these older methods, and in some cases, this may still work. These methods are viewed as being low-cost, easily transportable, easy to copy (i.e., back up), relatively flexible to use (e.g., write, draw, chart, etc.), and easy to sign and date.[1] However, paper notes aren't always easy to read, can be readily damaged or destroyed, can be difficult to search, require more manual time-consuming methods, and can be difficult to integrate with other paper-based data and information.[1]
Spreadsheets
As computing technology evolved and became more affordable, software makers had even more incentive to develop relevant and approachable software solutions to solve businesses' workflow challenges. Among these software solutions was the spreadsheet. Derived in concept from the paper-based ledgers accountants and traders would use, the electronic spreadsheet suddenly allowed businesses to perform calculations automatically, saving users time.[2] Laboratories picked up on this electronic, ledger-based approach to documenting experimental results and making routine analytical calculations. However, as labs of all types have fallen under greater scrutiny from regulators, the electronic spreadsheet method of documentation and calculation of analytical results shows inefficiencies and inadequacies, including difficulty in preventing changes to fields and maintaining an accurate representation of the who, what, when, and where of recorded values.
Databases
Databases also came into popularity with the advent of computing technology. Tabular and relational representation of data points, with the ability to assign labels to those data points, became useful for the electronic storage and retrieval of all sorts of data.[3] Like the spreadsheet, it's not surprising that some laboratories latched on to the idea of keeping track of experimental, analytical, and quality control data in a database. However, these are best used for structured data, and as electronic types of data and information have evolved into more sophisticated forms such as images, audio, and other unstructured formats, the traditional database has shown its weaknesses.
ERPs
Compared to spreadsheets and databases, the ERP is a bit more modern software solution, specifically designed to help streamline and unify business processes across an entire organization. These systems were originally designed for larger enterprises such as banks, manufacturers, and insurance companies, helping them manage finances, assets, inventory and supply chain, training, and other business aspects. However, even today most don't have the functionality necessary to handle more laboratory-specific activities, stretching the ERP's functionality to the limit. (There are examples of savvy software vendors who've built laboratory-specific modules for existing ERPs like open-source Odoo[4], but these appear to be few and far between.)
Deficiencies in most LIMS alternatives
To be sure, a LIMS is an investment for any sized laboratory, whether it's almost exclusively monetary (with some other organization doing a majority of the heavy lifting, as with a cloud-based solution) or some combination of monetary and in-house resource expenditure (as with a self-hosted solution located on-premises, whether that solution is a commercial proprietary offering or an open-source offering). Even an open-source LIMS still requires the lab to lean on an employee or third-party consultant to set up, configure, and maintain the software (or even modify the source code), as well as maintain the local IT infrastructure to support it. The open-source route may make sense for small, single labs with a couple of instruments, but the lack of regulatory-driven functionality like an audit trail in all but a few open-source LIMS (e.g., SENAITE[5]) may significantly restrict the available options to such labs.
This brings up the point of what a lab typically sacrifices with LIMS alternatives such as paper notebooks, spreadsheets, databases, and ERPs. These alternatives rarely address regulatory and/or internal need for[6][1][7][8][9][10][11]:
- ensuring analytical results haven't been maliciously or accidentally modified (such as with audit trails that clearly and properly maintain the metadata surrounding an inputted value, as well as any changes made to it);
- clearly and accurately documenting a wide variety of data and metadata about a given sample or analysis;
- ensuring software tools like spreadsheets are validated;
- ensuring data is contemporaneous (i.e., "current") and not fabricated post-analysis;
- maintaining data integrity beyond what audit trails provide;
- maintaining, archiving, and disposing of data and information for a designated period of time, whether it's paper or electronic;
- ensuring recorded values are treated uniformly for all lab operations, using the same units, rounding rules, formulas, limits, etc.;
- maintaining the security of proprietary lab data and information, including methods, analytical values, and associated reports;
- ensuring accurate and timely analytical results that have been officially validated/approved by one or more individuals (with that validation/approval getting properly documented);
- ensuring timely retrieval of data and information to more readily support decision-making and audits;
- allowing more than one user to access, add, and modify lab data and information;
- supporting more timely recording of analytical values from instruments; and
- supporting later conversion of paper-based data and information into structured, readable, and importable electronic formats.
This is not to say that paper-based laboratory notebooks, spreadsheets, databases, and ERPs can't work for small, lightly-regulated laboratories. However, as more laboratory activities across all industries gain more regulatory oversight, and as clientele of said labs increasingly demand more timely, accurate, and defensible analytical results, today's laboratories are under pressure to move beyond little print-out slips from instruments, paper notebooks, controlled worksheets, and non-validated software tools that aren't purpose-built for labs. That some LIMS vendors have recognized that not every lab needs a megalithic software solution—in turn offering slimmed-down, more affordable LIMS solutions—is even more encouraging for small labs that want to take the next step towards improved workflows and greater data integrity.
Conclusion
References
- ↑ 1.0 1.1 1.2 Liscouski, J. (November 2023). "LII:A Science Student's Guide to Laboratory Informatics". In Douglas, S.E.. LIMSwiki.org. https://www.limswiki.org/index.php/LII:A_Science_Student%27s_Guide_to_Laboratory_Informatics. Retrieved 13 December 2023.
- ↑ Meikle, H.. "The history of spreadsheets". Sheetgo Blog. https://blog.sheetgo.com/spreadsheets-tips/history-of-spreadsheets/. Retrieved 15 December 2023.
- ↑ Fortune, S. (17 December 2020). "A Brief History of Databases". CS403: Introduction to Modern Database Systems. Saylor Academy. https://learn.saylor.org/mod/page/view.php?id=21059. Retrieved 15 December 2023.
- ↑ "Odoo LIMS". LogicaSoft SPRL. https://www.logicasoft.eu/en_US/lims. Retrieved 15 December 2023.
- ↑ "SENAITE - Features". SENAITE Foundation. https://www.senaite.com/features. Retrieved 13 December 2023.
- ↑ Liscouski, J. (April 2022). "Improving Lab Systems: From Paper to Spreadsheets to LIMS" (PDF). LabLynx, Inc.. https://www.lablynx.com/wp-content/uploads/2023/03/Improving-Lab-Systems-From-Paper-to-Spreadsheets-to-LIMS.pdf. Retrieved 13 December 2023.
- ↑ Ferrero, M.S. (2007). "Chapter 8: GLP Documentation". In Weinberg, Sandy. Good laboratory practice regulations. Drugs and the pharmaceutical sciences (4th ed ed.). New York: Informa Healthcare. pp. 223–240. ISBN 9780849375842. https://books.google.com/books?id=JR5i0Nz5UWEC&pg=PA233.
- ↑ AlTarawneh, Ghada; Thorne, Simon (2017). "A Pilot Study Exploring Spreadsheet Risk in Scientific Research". arXiv. doi:10.48550/ARXIV.1703.09785. https://arxiv.org/abs/1703.09785.
- ↑ McDowall, R.D. (2020). "Are Spreadsheets a Fast Track to Regulatory Non-Compliance?". LCGC Europe 33 (9): 468–76. https://www.chromatographyonline.com/view/are-spreadsheets-a-fast-track-to-regulatory-non-compliance. Retrieved 13 December 2023.
- ↑ Ma, Ming-Wei; Gao, Xian-Shu; Zhang, Ze-Yu; Shang, Shi-Yu; Jin, Ling; Liu, Pei-Lin; Lv, Feng; Ni, Wei et al. (6 November 2023). "Extracting laboratory test information from paper-based reports" (in en). BMC Medical Informatics and Decision Making 23 (1): 251. doi:10.1186/s12911-023-02346-6. ISSN 1472-6947. PMC PMC10629084. PMID 37932733. https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-023-02346-6.
- ↑ McDowall. R.D., "Chapter 13. Get Rid of Paper: Why Electronic Processes are Better for Data Integrity" (in en), Data Integrity and Data Governance (Cambridge: Royal Society of Chemistry): 281–304, doi:10.1039/9781788013277-00281, ISBN 978-1-78801-281-2, http://ebook.rsc.org/?DOI=10.1039/9781788013277-00281