Journal:Autonomous experimental systems in materials science

Full article title	Autonomous experimental systems in materials science
Journal	Science and Technology of Advanced Materials: Methods
Author(s)	Ishizuki, Naoya; Shimizu, Ryota; Hitosugi, Taro
Author affiliation(s)	Tokyo Institute of Technology, The University of Tokyo
Primary contact	Email: hitosugi at g dot ecc dot u dash tokyo dot ac dot jp
Year published	2023
Volume and issue	3(1)
Article #	2197519
DOI	10.1080/27660400.2023.2197519
ISSN	2766-0400
Distribution license	Creative Commons Attribution 4.0 International
Website	https://www.tandfonline.com/doi/full/10.1080/27660400.2023.2197519
Download	https://www.tandfonline.com/doi/pdf/10.1080/27660400.2023.2197519 (PDF)

Abstract

The emergence of autonomous experimental systems (AESs) integrating machine learning (ML) and robots is ushering in a paradigm shift in materials science. Using computer algorithms and robots to decide and perform all experimental steps, these systems require no human intervention. A current direction focuses on discovering unexpected materials and theories with unconventional research approaches. This article reviews the latest achievements and discusses the impact of AESs, which will fundamentally change the way we understand research. Moreover, as AESs continue to develop, the need to think about the role of human researchers becomes more pressing. While ML and robotics can free us from the repetitive aspects of research, we need to understand the strengths and limitations of ML and robots and focus on how humans can perform higher creativity. In addition, we also discuss inventorship and authorship in the era of autonomous systems.

Keywords: autonomous experimental system, closed-loop, machine learning, robots, materials science, inventorship, authorship, human researcher, human’s role

Graphic abstract:

Introduction

The total number of all possible small organic molecules is estimated to be at least 10⁶⁰[1,2], and from that one can imagine a similarly large number of possible materials being derived using those molecules. This number helps illustrate the vastness of the materials search space, which must contain many materials that can help address current societal problems. In a way, the world of materials is a frontier for exploration, much like space or the deep sea.

How can we quickly and systematically find unexpected materials within this enormous search space? To this end, materials science needs a tool that can transcend the limits of human capabilities to serve as a materials explorer (Figure 1), akin to a spaceship or a deep-sea exploration vessel.

Figure 1. The vision of the materials explorer. The exploration involves an autonomous experimental system, materials informatics, and human researchers. The heart of the materials explorer is an autonomous experimental system based on machine learning and robots (green, orange, and blue). This system is imbued with the skills of experts and generates large amounts of experimental data that could not have been generated by human researchers (data-production factory). The data generated by the autonomous experimental system is then processed by machine learning and simulations to predict new materials (materials informatics). In addition, the system organizes the data and generates "materials maps" and models, facilitating knowledge creation by providing researchers a sharable big-picture view of unexpected materials, thereby accelerating materials development.

The core of such a materials explorer is the autonomous experimental system (AES) based on machine learning (ML) and robots (green, orange, and purple in Figure 1). Here, the term "autonomous" means that a computer algorithm decides the next experimental steps while robots perform all experimental steps. This approach, which involves no human intervention, is called the closed-loop experiment (Figure 2).

Figure 2. The concept of an autonomous experimental system (AES). The system autonomously synthesizes materials with optimal physical characteristics without human intervention. Autonomy leads to significant improvements: 1) fully digitalized experiments transforms all experimental parameters, including process conditions, into data; 2) removal of human error makes reproducibility reliable; and 3) implicit knowledge can be digitalized and embedded.

In general, new materials are sought in a multi-dimensional space by optimizing many relevant experimental parameters. Because of the vastness of the search space, the manual optimization of these parameters by individual researchers only produces incremental results that do not show the big picture. However, this problem is ideally suited for the AES to address. Figure 2 illustrates one such example. Here, based on the initial instructions, ML decides which compound to synthesize and feeds the corresponding directions to the robots; the robots synthesize, test, and report the results back to the algorithm, repeating the cycle until the desired result is obtained. This autonomous experimental approach drastically speeds up the materials exploration processes.

The concept of the materials explorer fundamentally changes the way we understand and conduct materials research, across three stages:

Stage 1: Optimization of the yield of target substances - Here, the target compound is known, but the optimum synthesis conditions are unknown. The target compound is decided by the human researchers before the experiment, and the AES quickly optimizes the synthesis conditions of the target compound within the search space specified by the human researchers. [3]

Stage 2: Finding new materials with desired properties - Here, the target physical properties are decided, but the compound possessing these properties is unknown. The AES quickly finds the best material within the search space specified by the human researchers. In contrast to Stage 1, it is the composition of the materials that is changed to find new compounds to meet the required physical properties. Materials with a variety of crystal structures and hierarchical structures are also explored.

Stage 3: Finding new materials or principles that no one has thought of before - Here, new materials, theories, and principles that are unexpected to researchers are discovered by combining the results from autonomous experiments, materials informatics, and human researchers (Figure 1). [4]

At present, the proofs of concepts of Stages 1 and 2 have been demonstrated in a variety of fields. [5–17] Expanding the application of AESs to a variety of experiments has become the next objective. Meanwhile, Stage 3 is rapidly advancing owing to the development of ML, robotics, and materials informatics. In this stage, it is critical to embed the researchers’ intuition and experience into the construction of theories. This inclusion of the human role is often referred to as having a "human-in-the-loop" or "researcher-in-the-loop."

An important point to note is that the transformation resulting from the three stages will change the way researchers think, giving completely new perspectives that researchers cannot obtain using conventional research methods. In addition, the transformation is not limited to a single laboratory; instead, it will change how we conduct research through a digital transformation of materials science. For example, experimentalists can remotely fabricate materials via the internet. In the same way, theorists can fabricate materials to test their predictions.

The data generated by such a system would fall into the domain of "big data." Researchers use this big data to extract human-readable information (i.e., via materials informatics). Because ML and robotics alone can neither find insights nor discover concepts in physics and chemistry, human researchers will always remain central to the research. The key point is that materials scientists must understand what ML and robotics can solve, and must set the right problem to be solved. The strength of human researchers lies in concept creation or problem identification in the larger context. Combining these strengths with ML and robotics is critical to accelerating research (i.e., the researcher-in-the-loop).

The process of autonomous materials exploration, however, raises one fundamental question: when experiments are performed autonomously and new materials are found without human intervention, who is the discoverer and inventor? This question not only relates to the authorship of papers and the inventorship of patents, but it also concerns researchers’ motivations. It is important to address this discussion as the autonomous experimental approach develops.

In this article, we review the trends and the prospects of AESs in materials science, specifically through the following aspects:

• the overall status of autonomous experiments in materials science;
• the history and the latest topics of autonomous materials synthesis;
• the future of research using the AES;
• some important take-home lessons we have learned when developing and using such a technology; and
• the future of authorship and inventorship.

Throughout this review, we address what human researchers should focus on in the era of autonomous research. The year 2020 was momentous for this era; there were significant advancements in the field of autonomous research. This review aims to further contribute to the expansion of the era of autonomous materials research.

Present status of autonomous experiments

A brief description of our own autonomous experiments

In the closed-loop cycle shown in Figure 2, researchers only need to choose the material properties to optimize and provide the system with the necessary raw materials; the automatic system then takes control, repeatedly synthesizing and measuring the properties of new compounds until the best one is found. The ML algorithm uses previous knowledge to decide how the synthesis conditions should be changed to approach the desired outcome with each cycle.

Recently, we have developed an autonomous synthesis of inorganic thin films. [3] In this proof-of-concept study, we demonstrated the autonomous fabrication of TiO₂ thin films with low resistance and showed that this system accelerates experiments by tenfold.

To obtain these results, we have used robotic modules of a sputter deposition apparatus and a robotic device for measuring resistance. Other modules with robotic synthesis and measurement equipment can be connected to this system to adapt to the desired research. The robotic arm transfers the samples from module to module as needed, and the Bayesian optimization algorithm predicts the synthesis parameters for the next iteration.

Autonomous experimental systems in the world

Recent years have witnessed the rapid progress of autonomous experiments, including a) development of ML technology, b) improvement of robot technology and expansion of its range of applications, and c) realization of autonomous experiments using ML and robots. As discussed in the introduction, examples of autonomous materials syntheses can be found to demonstrate Stages 1 and 2. [5–17] In this section, we review the history and the latest advances demonstrating these two stages (Table 1). [18–30]

Table 1. Examples of autonomous materials synthesis using robots. Research group (year) Robot mechanism Optimization objective/Synthesized material Variables/Algorithm Matsuda et al. (1988) [18] Robotic arm to manipulate the test tube Maximizing color-developing reactions for analysis (Stage 1) Optimizing the amount of reagent, reaction time, etc. using the simplex method Nikolaev et al. (2016) [19] Laser to perform both heating and spectroscopy Carbon nanotubes with maximized growth rate (Stage 1) Optimizing temperature, pressure, and gas composition using a genetic algorithm Christensen et al. (2021) [20] Liquid handling robot Maximizing the yield of Suzuki-Miyaura coupling reaction (Stage 1) Optimizing continuous variables (amount of ingredients, etc.) and categorical variables (catalyst type) using Bayesian optimization McMullen et al (2010) [21] Flow reactor (synthesis in liquid phase) Maximizing the yield of Heck reaction (Stage 1) Optimizing temperature and raw materials ratio using simplex method Sans et al. (2015) [22] Flow reactor Maximizing the yield of imine synthesis (Stage 1) Optimizing temperature and raw materials ratio using simplex method King et al. (2009) and King (2011) [23-24] Liquid handling robot, a robotic arm, etc. Discovering yeast genes (Stage 2) Generating hypotheses and experimentally testing these hypotheses Burger et al. (2020) [25] Free-roaming robot Photocatalyst mixtures with maximized photocatalytic activity (Stage 2) Optimizing the concentration of photocatalyst and additives using Bayesian optimization MacLeod et al. (2020) [26] A gripper, a pipette mount, and a robotic arm Organic hole transport materials with maximized hole mobility (Stage 2) Optimizing annealing times and dopant concentrations using Bayesian optimization Krishnadasan et al. (2007) [27] Flow reactor CdSe nanoparticles with targeted spectroscopic characteristics (Stage 2) Optimizing temperature and precursor concentrations using SNOBFIT method Tao et al. (2021) [28] Flow reactor Au nanoparticles with targeted spectroscopic characteristics (Stage 2) Optimizing reaction time and precursor concentrations using Bayesian optimization Desai et al. (2013) [29] Flow reactor Abl kinase inhibitors with maximum activity (Stage 2) Synthesizing from 27 × 10 row materials using random forest Dave et al. (2020) [30] Flow reactor Aqueous electrolytes with maximum electrochemical window (Stage 2) Optimizing each precursor solution volume using Bayesian optimization Shimizu et al. (2020) [3] A robot arm, a sputtering system, a resistance meter Nb-doped TiO2 thin film with minimized resistance (Stage 2) Optimizing oxygen partial pressure using Bayesian optimization

Stage 1: Optimization of the yield of target substances or reaction conditions

The idea of autonomous materials synthesis using robots has a long history. The first proposal of a fully automated closed-loop robot aiming for the optimization of chemical reaction parameters was published in 1978. [31] However, no experimental results were reported.

In the 1980s, Matsuda et al. reported the optimization of reaction conditions using an autonomous system. [18] The repetition of reagent adjustment, reaction, measurement, and prediction of the next experimental conditions based on the measured results were automatically performed. Here, test tubes were manipulated by a robot. The color-developing reactions for chemical analysis were optimized using the simplex method with three parameters: the amount of two reagents and the reaction time. The exhaustive grid search required 130 experiments, but the robotic system optimized the reaction in less than 28 experiments. While the purpose of this experiment was not to find a new compound but to optimize the color-developing reaction, it is a seminal pioneering work of autonomous experiments.

In 2016, Nikolaev et al. demonstrated an autonomous synthesis of inorganic materials. The authors synthesized carbon nanotubes using chemical vapor deposition. [19] They fabricated multiple columns containing a catalyst layer on a substrate in advance. Then, they heated the individual columns one by one with a laser while repeatedly moving the substrate to synthesize carbon nanotubes with different growth conditions. This heating laser was also used as an excitation source for Raman spectroscopy to observe the growth rate in situ. A genetic algorithm maximized the growth rate; the system optimized the temperature, pressure, and gas composition. Later the group utilized a Bayesian optimization as an optimization algorithm. [32]

For organics, Christensen et al. maximized the yield of the Suzuki-Miyaura coupling reaction using Bayesian optimization. [20] They used a commercially available liquid handling robot and ChemOS [33–35] for autonomous experiments. In the Bayesian optimization, Phoenics [36] and Gryffin [37] algorithms were used to optimize categorical variables (catalyst type) in addition to continuous variables (amount of catalyst, amount of feedstock, reaction temperature). Parallel autonomous process optimization experiments in batches were performed to shorten the time to complete the optimization.

The autonomous synthesis of organics using flow reactors was reported by McMullen et al. [21] The authors first reported on a Heck reaction, where the yield was maximized by optimizing the raw materials ratio and reaction time as independent variables, using the simplex method (Nelder-Mead method). Subsequently, the authors worked on other autonomous syntheses using flow reactors [38–39]. For example, they applied SNOBFIT (Stable Noisy Optimization by Branch and Fit) to a variety of reactions using a modular system. [40] In 2015, Sans et al. optimized the yield of an imine synthesis. [22] The authors also used the simplex method to tune the raw materials ratio and reaction time. The system was equipped with in-line nuclear magnetic resonance spectroscopy. The group also developed an organic synthesis robot that navigates chemical reaction spaces. [41–43]

Stage 2: Finding new materials with desired properties

In 2009, King et al. reported a seminal work [23–24], where a robot "Adam" autonomously generated functional genomics hypotheses and experimentally tested the hypotheses using robots. The system measured the growth curves of selected microbial strains growing in defined media, "discovering" three novel yeast genes. The equipment comprised a liquid handling robot, a robot arm, and an incubator, all fixed in one place.

In 2020, Burger et al. demonstrated a free-roaming robot that moved around the laboratory to perform autonomous experiments using the same equipment as those used by its human counterparts. [25] The robot aimed to maximize photocatalytic activity by optimizing the concentrations of photocatalyst and additives. Based on Bayesian optimization, the robot identified the photocatalyst mixtures that were six times more active than the initial formulation. The robot completed 688 experiments in eight days. This number of experiments would take a human researcher several months. For reagent weighing, the robot handled both powder and liquid materials.

In the same year, MacLeod et al. reported the autonomous fabrication of organic thin films. [26] A robot "Ada," equipped with a gripper, a pipette mount, and a robotic arm for liquid injection and substrate transfer, maximized the hole mobility of organic hole transport materials used in perovskite solar cells. The dopant concentrations and annealing time were optimized using Bayesian optimization. Ada finished the experiment in five days instead of nine months. Later, the group used Ada to define a Pareto front of conductivities and processing temperatures for palladium films formed by combustion synthesis. [44] There are other liquid handling robots developed for closed-loop, e.g., Wu et al. reported an automated platform for organic laser discovery, including not only synthesis and compound identification but also integrated target property characterization. [45]

There are several autonomous experiments based on flow reactors (synthesis in the liquid phase). For inorganic nanoparticles, Krishnadasan et al. autonomously synthesized CdSe nanoparticles with optimized intensity for a chosen emission wavelength in 2007 (SNOBFIT method). [27] Later, other groups reported various autonomous experiments on inorganic nanoparticles. [46–50]. For example, Tao et al. synthesized Au nanoparticles [28] with targeted spectroscopic characteristics using the above-mentioned Gryffin [37] in 2021.

For organic compounds, Desai et al. synthesized Abl kinase inhibitors with maximum activity in 2013. [29] Out of 270 possible combinations (10 types × 27 types) from the raw materials, the system found the novel Abl kinase inhibitor after synthesizing only 21 compounds. The prediction model used random forest regression to handle the types of raw materials with different structures.

Dave et al. reported isolating aqueous electrolytes with the maximum electrochemical window in 2020. [30] The authors used Bayesian optimization to optimize the solution volume of three aqueous Li salts or four aqueous Na salts. The results indicated that non-smooth chemical responses are observed along the axis of the amount of NaBr. The authors pointed out that log-scaling, which varies rapidly with quantity, such as the amount of NaBr, gives the response surface both smaller gradients along this axis and a much-improved performance because Gaussian process regression requires an assumption of smoothness on the response surface. They also pointed out that the selection and presentation of the design space for materials search are of utmost importance to autonomous task design.

Future of Stages 1 and 2: Autonomously search within the preset search space

Currently, Stages 1 and 2 are not widespread because the range of applicable experiments is still limited. As more sophisticated robots are developed, the application of autonomous research will expand in turn. The problem lies in that conventional robots are rigidly designed for precision, aiming for zero error. Because they are programmed based on if-then statements, these robots cannot respond flexibly to changing situations. Therefore, it is difficult to implement the tacit knowledge of skilled human researchers in robot systems. With soft control, soft structure, multiple sensing devices, and ML, a more flexible robot can absorb errors and disruptions, and can operate without stoppage or breakdowns. [51—52]. Thus, developing robot technology that can respond flexibly until the goal is achieved is important. To this end, it is essential to develop new techniques for robots to learn the minute details of any required motions.

When versatile robots capable of performing a variety of experiments become affordable, the introduction of robots will rapidly expand into many research fields. At the time of this writing, the cost of robots is reducing rapidly to the point where labs can consider using such robot-experiment automation. However, more effort is needed to help materials scientists implement robots into their experiments.

We note that not all researchers will adopt this style of research. The conventional research approach—which relies on researchers’ experience, knowledge, and intuition—will remain essential for conceiving new synthetic methods and analytical techniques. We expect there will be a hybrid between conventional and autonomous approaches.

Autonomous experimental systems are not only useful for performing high-throughput experiments; the most critical aim is to help human researchers deepen their research. Human researchers need to think about how to analyze the collected data, how the new materials can be used, and what theories can be deduced from unexpected results. AESs free human researchers from repetitive tasks and enable researchers to conduct innovative research. Furthermore, combining the system with advanced materials prediction based on computer simulation is the step that enables human researchers to perform more creative work (Figure 3).

Figure 3. The human researchers will always play the leading role. The autonomous laboratory is for researchers. This technology makes it possible to narrow down candidate materials with high predictive performance and to pinpoint materials synthesis experiments. The researchers then work on highly creative tasks and acquire "tacit knowledge" of an even higher level. In turn, the new knowledge is digitized again. It is essential to repeat this cycle.

With the implementation of autonomous experimental systems, theorists can test their original ideas. Once theorists predict a promising material, they can log on to the AES via the internet and synthesize the new material. Currently, we are developing a system capable of automatically synthesizing materials from simple instructions, such as "synthesize AxByCz with a crystal structure of D." The system automatically measures X-ray diffraction patterns and feeds the results back to the algorithm to predict the next synthesis condition.

Stage 3: Finding materials that no one has thought of before

Because the AES will generate increasingly larger amounts of data, it is essential to combine this type of system with materials informatics and computer simulations to extend human thinking. In so doing, the generated data will promote the emergence of unexpected results that are beyond conventional theories and experiences. It is also important to embed the researchers’ intuition and experience, which are tacit knowledge, into the autonomous system and share them with other researchers (Figure 4).

Figure 4. A vision of how research will be conducted. In the conventional way of research, there has been a disruption between researchers and robots/artificial intelligence (AI). In the near future, researchers will develop materials based on an enormous amount of experimental and simulation results. In this "researcher-in-the-loop" process, human researchers will obtain new insights, a bird’s eye view of materials through multifaceted materials characterization and resulting materials big data.

The following are some of the remaining challenges.

Increasing the amount of reliable materials data (materials checkup system and measurement)

To date, materials informatics is still limited by the lack of a reliable materials experiment database. Such a database cannot be manually acquired. To this end, AESs enable the storage of all data, ranging from the fabrication processes (synthesis conditions and experimental environment) to the structural and composition information, to the measured materials properties. Even the negative data (data that do not show the desired values) are stored for later use. This inclusive approach to data storage drastically increases the amount of data available for accurate prediction based on ML. In addition, all data are highly reproducible because they are not acquired by humans (i.e., devoid of human errors). Therefore, it is possible to analyze in detail which process parameters are correlated with physical properties, leading to technology for predicting synthetic processes (i.e., process informatics).

The sheer increase in experimental speed directly translates to an increase in the amount of materials data. The speed of autonomous experiments is estimated to be at least one order of magnitude faster than conventional experiments; it is possible to accelerate the acquisition of measurement data by another order of magnitude using the materials checkup system (Figure 5). [3]

Figure 5. Concept of the materials checkup system for finding unexpected results. All the material-property measurements are done automatically. Original source Shimizu et al. 2020, reprinted with permission under a Creative Commons Attribution (CC BY) license. [3]

In conventional research, human researchers often need to transport their samples to multiple analysis systems to evaluate the physical properties. To avoid errors associated with sample transport and shorten the time for analysis, all analytical modules should be connected to each other. Connecting multiple materials property measurement devices and automatically measuring one after another enable multifaceted materials characterization (Figure 5). With the recent development of measurement and analysis informatics [53], it is possible to perform measurements with even higher accuracy, precision, and sensitivity measurements in a short period of time. [54]

The key to the multifaceted evaluation is to increase the probability of finding unexpected results. We have already experienced finding an unexpected electrode material using multifaceted evaluation while targeting the synthesis of a solid electrolyte.

Constructing scientific theories from a bird’s-eye view

The AES does not search aimlessly in the search space. Instead, it creates a materials map [55] based on the large amount of data described in the previous section to give a bird’s-eye view of the materials world (Figure 6).

Figure 6. Creation of a "materials map" to identify unexplored materials. Autonomous experiments enable the creation of materials maps using experimental data. We put particular emphasis on unexpected experimental results and explore their surroundings.

Specifically, materials informatics methods help organize the enormous amount of multi-dimensional data (e.g., actual experimental data and high-throughput first-principles simulations) into human-readable forms. This data is overlaid with previously reported experimental and simulated data to create a materials map.

The materials map depicts the untapped and promising areas that should be experimentally verified. This depiction stimulates the researchers’ thinking in ways conventional research methods cannot. The new ideas can then be immediately tested using the AES (Figure 1).

The unveiling of the hidden correlations and relationships between materials properties and processes will promote the construction of new theories. Until now, completely unexpected and never-before-observed results are often ignored because it has been difficult to distinguish whether they are real or whether they are caused by experimental error. However, these results can be reproduced when using the AES; unexpected results (i.e., "outliers") will not be ignored, making it possible to create new scientific theories that explain them (Figure 7).

Figure 7. Importance of outliers. Suppose there is a number hidden in the tiles in (a). With no data, nothing can be predicted. (b) With some amount of data, hypotheses can be proposed. Typical researchers could not predict that the number "1" is hidden in this figure. Only an exceptional researcher can recognize the hidden "1." Materials science to date has been "local" (i.e., belonging to particular researchers) in this way. (c) When big data is available, researchers can easily recognize "1," without relying on such exceptional researchers. Furthermore, by considering the "outliers" (shown in red), we can see the true situation: a "4" rather than a "1." We can thus construct new theories (i.e., big-picture materials science).

Lastly, to overview a massive amount of materials data, it is necessary to have the technology capable of converting this data for humans to understand. In particular, the development of technologies for explainable ML and visualization of information in a multi-dimensional space through dimensionality reduction is mandatory.

Important points for the actual operation of an autonomous system

This section discusses some important take-home lessons we have learned from our constructing and operating the AES.

Notes

This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added. In the original, there are multiple instances of citing research work using the last name of the last author listed, rather than the last name of the first author listed; this may have been a product of Japanese culture tending to read text from right to left. For this version, the last name of the first author was used to be consistent with research norms.