Journal:Autonomous experimental systems in materials science

Full article title	Autonomous experimental systems in materials science
Journal	Science and Technology of Advanced Materials: Methods
Author(s)	Ishizuki, Naoya; Shimizu, Ryota; Hitosugi, Taro
Author affiliation(s)	Tokyo Institute of Technology, The University of Tokyo
Primary contact	Email: hitosugi at g dot ecc dot u dash tokyo dot ac dot jp
Year published	2023
Volume and issue	3(1)
Article #	2197519
DOI	10.1080/27660400.2023.2197519
ISSN	2766-0400
Distribution license	Creative Commons Attribution 4.0 International
Website	https://www.tandfonline.com/doi/full/10.1080/27660400.2023.2197519
Download	https://www.tandfonline.com/doi/pdf/10.1080/27660400.2023.2197519 (PDF)

Abstract

The emergence of autonomous experimental systems (AESs) integrating machine learning (ML) and robots is ushering in a paradigm shift in materials science. Using computer algorithms and robots to decide and perform all experimental steps, these systems require no human intervention. A current direction focuses on discovering unexpected materials and theories with unconventional research approaches. This article reviews the latest achievements and discusses the impact of AESs, which will fundamentally change the way we understand research. Moreover, as AESs continue to develop, the need to think about the role of human researchers becomes more pressing. While ML and robotics can free us from the repetitive aspects of research, we need to understand the strengths and limitations of ML and robots and focus on how humans can perform higher creativity. In addition, we also discuss inventorship and authorship in the era of autonomous systems.

Keywords: autonomous experimental system, closed-loop, machine learning, robots, materials science, inventorship, authorship, human researcher, human’s role

Graphic abstract:

Introduction

The total number of all possible small organic molecules is estimated to be at least 10⁶⁰[1,2], and from that one can imagine a similarly large number of possible materials being derived using those molecules. This number helps illustrate the vastness of the materials search space, which must contain many materials that can help address current societal problems. In a way, the world of materials is a frontier for exploration, much like space or the deep sea.

How can we quickly and systematically find unexpected materials within this enormous search space? To this end, materials science needs a tool that can transcend the limits of human capabilities to serve as a materials explorer (Figure 1), akin to a spaceship or a deep-sea exploration vessel.

Figure 1. The vision of the materials explorer. The exploration involves an autonomous experimental system, materials informatics, and human researchers. The heart of the materials explorer is an autonomous experimental system based on machine learning and robots (green, orange, and blue). This system is imbued with the skills of experts and generates large amounts of experimental data that could not have been generated by human researchers (data-production factory). The data generated by the autonomous experimental system is then processed by machine learning and simulations to predict new materials (materials informatics). In addition, the system organizes the data and generates "materials maps" and models, facilitating knowledge creation by providing researchers a sharable big-picture view of unexpected materials, thereby accelerating materials development.

The core of such a materials explorer is the autonomous experimental system (AES) based on machine learning (ML) and robots (green, orange, and purple in Figure 1). Here, the term "autonomous" means that a computer algorithm decides the next experimental steps while robots perform all experimental steps. This approach, which involves no human intervention, is called the closed-loop experiment (Figure 2).

Figure 2. The concept of an autonomous experimental system (AES). The system autonomously synthesizes materials with optimal physical characteristics without human intervention. Autonomy leads to significant improvements: 1) fully digitalized experiments transforms all experimental parameters, including process conditions, into data; 2) removal of human error makes reproducibility reliable; and 3) implicit knowledge can be digitalized and embedded.

In general, new materials are sought in a multi-dimensional space by optimizing many relevant experimental parameters. Because of the vastness of the search space, the manual optimization of these parameters by individual researchers only produces incremental results that do not show the big picture. However, this problem is ideally suited for the AES to address. Figure 2 illustrates one such example. Here, based on the initial instructions, ML decides which compound to synthesize and feeds the corresponding directions to the robots; the robots synthesize, test, and report the results back to the algorithm, repeating the cycle until the desired result is obtained. This autonomous experimental approach drastically speeds up the materials exploration processes.

The concept of the materials explorer fundamentally changes the way we understand and conduct materials research, across three stages:

Stage 1: Optimization of the yield of target substances - Here, the target compound is known, but the optimum synthesis conditions are unknown. The target compound is decided by the human researchers before the experiment, and the AES quickly optimizes the synthesis conditions of the target compound within the search space specified by the human researchers. [3]

Stage 2: Finding new materials with desired properties - Here, the target physical properties are decided, but the compound possessing these properties is unknown. The AES quickly finds the best material within the search space specified by the human researchers. In contrast to Stage 1, it is the composition of the materials that is changed to find new compounds to meet the required physical properties. Materials with a variety of crystal structures and hierarchical structures are also explored.

Stage 3: Finding new materials or principles that no one has thought of before - Here, new materials, theories, and principles that are unexpected to researchers are discovered by combining the results from autonomous experiments, materials informatics, and human researchers (Figure 1). [4]

At present, the proofs of concepts of Stages 1 and 2 have been demonstrated in a variety of fields. [5–17] Expanding the application of AESs to a variety of experiments has become the next objective. Meanwhile, Stage 3 is rapidly advancing owing to the development of ML, robotics, and materials informatics. In this stage, it is critical to embed the researchers’ intuition and experience into the construction of theories. This inclusion of the human role is often referred to as having a "human-in-the-loop" or "researcher-in-the-loop."

An important point to note is that the transformation resulting from the three stages will change the way researchers think, giving completely new perspectives that researchers cannot obtain using conventional research methods. In addition, the transformation is not limited to a single laboratory; instead, it will change how we conduct research through a digital transformation of materials science. For example, experimentalists can remotely fabricate materials via the internet. In the same way, theorists can fabricate materials to test their predictions.

The data generated by such a system would fall into the domain of "big data." Researchers use this big data to extract human-readable information (i.e., via materials informatics). Because ML and robotics alone can neither find insights nor discover concepts in physics and chemistry, human researchers will always remain central to the research. The key point is that materials scientists must understand what ML and robotics can solve, and must set the right problem to be solved. The strength of human researchers lies in concept creation or problem identification in the larger context. Combining these strengths with ML and robotics is critical to accelerating research (i.e., the researcher-in-the-loop).

The process of autonomous materials exploration, however, raises one fundamental question: when experiments are performed autonomously and new materials are found without human intervention, who is the discoverer and inventor? This question not only relates to the authorship of papers and the inventorship of patents, but it also concerns researchers’ motivations. It is important to address this discussion as the autonomous experimental approach develops.

In this article, we review the trends and the prospects of AESs in materials science, specifically through the following aspects:

• the overall status of autonomous experiments in materials science;
• the history and the latest topics of autonomous materials synthesis;
• the future of research using the AES;
• some important take-home lessons we have learned when developing and using such a technology; and
• the future of authorship and inventorship.

Throughout this review, we address what human researchers should focus on in the era of autonomous research. The year 2020 was momentous for this era; there were significant advancements in the field of autonomous research. This review aims to further contribute to the expansion of the era of autonomous materials research.

Present status of autonomous experiments

A brief description of our own autonomous experiments

In the closed-loop cycle shown in Figure 2, researchers only need to choose the material properties to optimize and provide the system with the necessary raw materials; the automatic system then takes control, repeatedly synthesizing and measuring the properties of new compounds until the best one is found. The ML algorithm uses previous knowledge to decide how the synthesis conditions should be changed to approach the desired outcome with each cycle.

Recently, we have developed an autonomous synthesis of inorganic thin films. [3] In this proof-of-concept study, we demonstrated the autonomous fabrication of TiO₂ thin films with low resistance and showed that this system accelerates experiments by tenfold.

To obtain these results, we have used robotic modules of a sputter deposition apparatus and a robotic device for measuring resistance. Other modules with robotic synthesis and measurement equipment can be connected to this system to adapt to the desired research. The robotic arm transfers the samples from module to module as needed, and the Bayesian optimization algorithm predicts the synthesis parameters for the next iteration.

Autonomous experimental systems in the world

Recent years have witnessed the rapid progress of autonomous experiments, including a) development of ML technology, b) improvement of robot technology and expansion of its range of applications, and c) realization of autonomous experiments using ML and robots. As discussed in the introduction, examples of autonomous materials syntheses can be found to demonstrate Stages 1 and 2. [5–17] In this section, we review the history and the latest advances demonstrating these two stages (Table 1). [18–30]

Table 1. Examples of autonomous materials synthesis using robots. Research group (year) Robot mechanism Optimization objective/Synthesized material Variables/Algorithm Matsuda et al. (1988) [18] Robotic arm to manipulate the test tube Maximizing color-developing reactions for analysis (Stage 1) Optimizing the amount of reagent, reaction time, etc. using the simplex method Nikolaev et al. (2016) [19] Laser to perform both heating and spectroscopy Carbon nanotubes with maximized growth rate (Stage 1) Optimizing temperature, pressure, and gas composition using a genetic algorithm Christensen et al. (2021) [20] Liquid handling robot Maximizing the yield of Suzuki-Miyaura coupling reaction (Stage 1) Optimizing continuous variables (amount of ingredients, etc.) and categorical variables (catalyst type) using Bayesian optimization McMullen et al (2010) [21] Flow reactor (synthesis in liquid phase) Maximizing the yield of Heck reaction (Stage 1) Optimizing temperature and raw materials ratio using simplex method Sans et al. (2015) [22] Flow reactor Maximizing the yield of imine synthesis (Stage 1) Optimizing temperature and raw materials ratio using simplex method King et al. (2009) and King (2011) [23-24] Liquid handling robot, a robotic arm, etc. Discovering yeast genes (Stage 2) Generating hypotheses and experimentally testing these hypotheses Burger et al. (2020) [25] Free-roaming robot Photocatalyst mixtures with maximized photocatalytic activity (Stage 2) Optimizing the concentration of photocatalyst and additives using Bayesian optimization MacLeod et al. (2020) [26] A gripper, a pipette mount, and a robotic arm Organic hole transport materials with maximized hole mobility (Stage 2) Optimizing annealing times and dopant concentrations using Bayesian optimization Krishnadasan et al. (2007) [27] Flow reactor CdSe nanoparticles with targeted spectroscopic characteristics (Stage 2) Optimizing temperature and precursor concentrations using SNOBFIT method Tao et al. (2021) [28] Flow reactor Au nanoparticles with targeted spectroscopic characteristics (Stage 2) Optimizing reaction time and precursor concentrations using Bayesian optimization Desai et al. (2013) [29] Flow reactor Abl kinase inhibitors with maximum activity (Stage 2) Synthesizing from 27 × 10 row materials using random forest Dave et al. (2020) [30] Flow reactor Aqueous electrolytes with maximum electrochemical window (Stage 2) Optimizing each precursor solution volume using Bayesian optimization Shimizu et al. (2020) [3] A robot arm, a sputtering system, a resistance meter Nb-doped TiO2 thin film with minimized resistance (Stage 2) Optimizing oxygen partial pressure using Bayesian optimization

Stage 1: Optimization of the yield of target substances or reaction conditions

The idea of autonomous materials synthesis using robots has a long history. The first proposal of a fully automated closed-loop robot aiming for the optimization of chemical reaction parameters was published in 1978. [31] However, no experimental results were reported.

In the 1980s, Matsuda et al. reported the optimization of reaction conditions using an autonomous system. [18] The repetition of reagent adjustment, reaction, measurement, and prediction of the next experimental conditions based on the measured results were automatically performed. Here, test tubes were manipulated by a robot. The color-developing reactions for chemical analysis were optimized using the simplex method with three parameters: the amount of two reagents and the reaction time. The exhaustive grid search required 130 experiments, but the robotic system optimized the reaction in less than 28 experiments. While the purpose of this experiment was not to find a new compound but to optimize the color-developing reaction, it is a seminal pioneering work of autonomous experiments.

In 2016, Nikolaev et al. demonstrated an autonomous synthesis of inorganic materials. The authors synthesized carbon nanotubes using chemical vapor deposition. [19] They fabricated multiple columns containing a catalyst layer on a substrate in advance. Then, they heated the individual columns one by one with a laser while repeatedly moving the substrate to synthesize carbon nanotubes with different growth conditions. This heating laser was also used as an excitation source for Raman spectroscopy to observe the growth rate in situ. A genetic algorithm maximized the growth rate; the system optimized the temperature, pressure, and gas composition. Later the group utilized a Bayesian optimization as an optimization algorithm. [32]

For organics, Christensen et al. maximized the yield of the Suzuki-Miyaura coupling reaction using Bayesian optimization. [20] They used a commercially available liquid handling robot and ChemOS [33–35] for autonomous experiments. In the Bayesian optimization, Phoenics [36] and Gryffin [37] algorithms were used to optimize categorical variables (catalyst type) in addition to continuous variables (amount of catalyst, amount of feedstock, reaction temperature). Parallel autonomous process optimization experiments in batches were performed to shorten the time to complete the optimization.

The autonomous synthesis of organics using flow reactors was reported by McMullen et al. [21] The authors first reported on a Heck reaction, where the yield was maximized by optimizing the raw materials ratio and reaction time as independent variables, using the simplex method (Nelder-Mead method). Subsequently, the authors worked on other autonomous syntheses using flow reactors [38–39]. For example, they applied SNOBFIT (Stable Noisy Optimization by Branch and Fit) to a variety of reactions using a modular system. [40] In 2015, Sans et al. optimized the yield of an imine synthesis. [22] The authors also used the simplex method to tune the raw materials ratio and reaction time. The system was equipped with in-line nuclear magnetic resonance spectroscopy. The group also developed an organic synthesis robot that navigates chemical reaction spaces. [41–43]

Stage 2: Finding new materials with desired properties

In 2009, King et al. reported a seminal work [23–24], where a robot "Adam" autonomously generated functional genomics hypotheses and experimentally tested the hypotheses using robots. The system measured the growth curves of selected microbial strains growing in defined media, "discovering" three novel yeast genes. The equipment comprised a liquid handling robot, a robot arm, and an incubator, all fixed in one place.

In 2020, Burger et al. demonstrated a free-roaming robot that moved around the laboratory to perform autonomous experiments using the same equipment as those used by its human counterparts. [25] The robot aimed to maximize photocatalytic activity by optimizing the concentrations of photocatalyst and additives. Based on Bayesian optimization, the robot identified the photocatalyst mixtures that were six times more active than the initial formulation. The robot completed 688 experiments in eight days. This number of experiments would take a human researcher several months. For reagent weighing, the robot handled both powder and liquid materials.

In the same year, MacLeod et al. reported the autonomous fabrication of organic thin films. [26] A robot "Ada," equipped with a gripper, a pipette mount, and a robotic arm for liquid injection and substrate transfer, maximized the hole mobility of organic hole transport materials used in perovskite solar cells. The dopant concentrations and annealing time were optimized using Bayesian optimization. Ada finished the experiment in five days instead of nine months. Later, the group used Ada to define a Pareto front of conductivities and processing temperatures for palladium films formed by combustion synthesis. [44] There are other liquid handling robots developed for closed-loop, e.g., Wu et al. reported an automated platform for organic laser discovery, including not only synthesis and compound identification but also integrated target property characterization. [45]

There are several autonomous experiments based on flow reactors (synthesis in the liquid phase). For inorganic nanoparticles, Krishnadasan et al. autonomously synthesized CdSe nanoparticles with optimized intensity for a chosen emission wavelength in 2007 (SNOBFIT method). [27] Later, other groups reported various autonomous experiments on inorganic nanoparticles. [46–50]. For example, Tao et al. synthesized Au nanoparticles [28] with targeted spectroscopic characteristics using the above-mentioned Gryffin [37] in 2021.

For organic compounds, Desai et al. synthesized Abl kinase inhibitors with maximum activity in 2013. [29] Out of 270 possible combinations (10 types × 27 types) from the raw materials, the system found the novel Abl kinase inhibitor after synthesizing only 21 compounds. The prediction model used random forest regression to handle the types of raw materials with different structures.

Dave et al. reported isolating aqueous electrolytes with the maximum electrochemical window in 2020. [30] The authors used Bayesian optimization to optimize the solution volume of three aqueous Li salts or four aqueous Na salts. The results indicated that non-smooth chemical responses are observed along the axis of the amount of NaBr. The authors pointed out that log-scaling, which varies rapidly with quantity, such as the amount of NaBr, gives the response surface both smaller gradients along this axis and a much-improved performance because Gaussian process regression requires an assumption of smoothness on the response surface. They also pointed out that the selection and presentation of the design space for materials search are of utmost importance to autonomous task design.

Notes

This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added. In the original, there are multiple instances of citing research work using the last name of the last author listed, rather than the last name of the first author listed; this may have been a product of Japanese culture tending to read text from right to left. For this version, the last name of the first author was used to be consistent with research norms.