Scientific data management system

NIST tests standard interfaces for its lab instruments. SDMSs allow labs to integrate raw and processed instrument data with other types of data, unstructured and structured.

A scientific data management system (SDMS) (occasionally referenced to as a laboratory data management system [LDMS]) is software that acts similarly to a document management system (DMS), capturing, cataloging, and archiving data generated by laboratory instruments (e.g., high-performance liquid chromatography and mass spectrometry instruments) and applications (e.g., laboratory information management systems, electronic laboratory notebooks, and other analytical applications) in a compliant, often pre-defined manner best suitable for its intended use, whether it be structured, unstructured, or semi-structured data. The SDMS can also act as a gatekeeper, serving platform-independent data to informatics applications and other stakeholders.

Purpose and technology

An SDMS is used to improve data handling and management issues in a number of scientific disciplines. As the four Vs of modern big data—volume, variety, veracity, and velocity—increase time spent on data acquisition and management, taking time away from other aspects of scientific research and complicating aspects of experimental reproducibility, solutions like an SDMS can help better manage the total lifecycle of data. This is accomplished through a variety of tools, including data normalization and integration, data sharing and management, metadata capture and management, data object and record management, and robust search tools.

As with many other laboratory informatics tools, the lines between an SDMS, LIMS, ELN, and other systems are at times blurred, as functionality from these systems makes their way into each other. However, there are some essential qualities that an SDMS owns that distinguishes it from other informatics systems:

1. While a LIMS has traditionally been built to handle structured, mostly homogeneous data, an SDMS (and systems like it) is built to handle unstructured, mostly heterogeneous data, though many can handle structured, unstructured, and semi-structured data.

2. An SDMS typically acts as a seamless "wrapper" for other data systems like LIMS and ELN in the laboratory, though sometimes the SDMS software is readily apparent.

3. An SDMS is designed primarily for data consolidation and reuse, knowledge integration and management, and knowledge asset discovery and realization.

An SDMS can be seen as one potential solution for handling unstructured data, which can make up nearly 75 percent of a research and development unit's data. This includes PDF files, images, instrument data, spreadsheets, and other forms of data rendered in many environments in the laboratory. Traditional SDMSs have focused on acting as a nearly invisible blanket or wrapper that integrate information from corporate offices (standard operating procedures, safety documents, etc.) with data from lab devices and other data management tools, all to be indexed and searchable from a central database. An SDMS also must be focused on increasing research productivity without sacrificing data sharing and collaboration efforts.

Some of the things a standard SDMS may be asked to do include, but are not limited to:

• store and archive raw data files;
• interact real-time with simple and complex laboratory instruments;
• retrieve worklists from LIMS and convert them to sequence files;
• require review and approval of actions, with electronic signatures;
• capture provenance information of data;
• allow for annotation of records and collections to inform or warn other users of relevant changes or errors;
• analyze and create reports on laboratory instrument functions;
• perform complex calculations and comparisons of two different sample groups;
• monitor environmental conditions and react when base operating parameters are out of range;
• act as an operational database that allows selective importation/exportation of ELN data;
• manage workflows based on data imported into the SDMS;
• validate other computer systems and software in the laboratory; and
• identify and retrieve data and metadata useful for training artificial intelligence and machine learning agents.

Further reading

• Stansberry, Dale; Somnath, Suhas; Breet, Jessica; Shutt, Gregory; Shankar, Mallikarjun (1 December 2019). "DataFed: Towards Reproducible Research via Federated Data Management". 2019 International Conference on Computational Science and Computational Intelligence (CSCI) (Las Vegas, NV, USA: IEEE): 1312–1317. doi:10.1109/CSCI49370.2019.00245. ISBN 978-1-7281-5584-5. https://ieeexplore.ieee.org/document/9071425/.