Bayesian optimization

1. Identify the objective function(s) and decision variables (with bounds).
2. A space-filling design (e.g. Latin hypercube sample) is generated and experiments are performed.
3. The surrogate model is (re)trained using available experimental data.
4. Using the surrogate model, the acquisition function is maximized to recommend the next experiment(s).
5. Experiment(s) are performed and added to the training data.
6. If the goal has not been attained and the experimental budget has not been exhausted, GOTO step 3.

 

Bayesian optimization for chemical products and functional materials

The design of chemical-based products and functional materials is vital to modern technologies, yet remains expensive and slow. Artificial intelligence and machine learning offer new approaches to leverage data to overcome these challenges. This review focuses on recent applications of Bayesian optimization (BO) to chemical products and materials including molecular design, drug discovery, molecular modeling, electrolyte design, and additive manufacturing. Numerous examples show how BO often requires an order of magnitude fewer experiments than the Edisonian search. The essential equations for BO are introduced in a self-contained primer specifically written for chemical engineers and others new to the area. Finally, the review discusses four current research directions for BO and their relevance to product and materials design.

Machine learning-assisted ultrafast flash sintering of high-performance and flexible silver–selenide thermoelectric devices

Flexible thermoelectric generators (TEGs) have shown immense potential for serving as a power source for wearable electronics and the Internet of Things. A key challenge preventing large-scale application of TEGs lies in the lack of a high-throughput processing method, which can sinter thermoelectric (TE) materials rapidly while maintaining their high thermoelectric properties. Herein, we integrate high-throughput experimentation and Bayesian optimization (BO) to accelerate the discovery of the optimum sintering conditions of silver–senenide TE films using an ultrafast intense pulsed light (flash) sintering technique. Due to the nature of the high-dimensional optimization problem of flash sintering processes, a Gaussian process regression (GPR) machine learning model is established to rapidly recommend the optimum flash sintering variables based on Bayesian expected improvement. For the first time, an ultrahigh-power factor flexible TE film (a power factor of 2205 μW m⁻¹ K⁻²with a zT of 1.1 at 300 K) is demonstrated with a sintering time less than 1.0 second, which is several orders of magnitude shorter than that of conventional thermal sintering techniques. The films also show excellent flexibility with 92% retention of the power factor (PF) after 10³ bending cycles with a 5 mm bending radius. In addition, a wearable thermoelectric generator based on the flash-sintered films generates a very competitive power density of 0.5 mW cm⁻² at a temperature difference of 10 K. This work not only shows the tremendous potential of high-performance and flexible silver–selenide TEGs but also demonstrates a machine learning-assisted flash sintering strategy that could be used for ultrafast, high-throughput and scalable processing of functional materials for a broad range of energy and electronic applications.

When physics-informed data analytics outperforms black-box machine learning: A case study in thickness control for additive manufacturing

Aerosol jet printing (AJP) has emerged as a promising noncontact additive manufacturing method for high-resolution printing for a wide range of material systems. A key challenge limiting the broader adoption of AJP in the material science community is the lack of methods to precisely control thickness. Herein, we develop a model-based design of experiment (MBDoE) framework that integrates physics-informed models, nonlinear regression, and information criteria to postulate, select and calibrate the best model to describe and optimize the AJP manufacturing process. Starting with already available data from system commissioning (e.g., prior single variable sensitivity analysis), four candidate physics-informed models are postulated and trained. MBDoE identifies a single additional optimal experiment to validate these predictive models with quantified uncertainties, which are then used to determine the best experimental conditions to control printed film thickness. As a comparative benchmark, the analysis is repeated using the same dataset with nonparametric Gaussian process regression (GPR) model that does not incorporate physical information. Using MBDoE principles, we find that only five experiments are necessary to calibrate the nonlinear physics-informed parametric model, and with said limited data, this model outperforms the black-box machine learning GPR model. This key result underscores an emerging trend in the data science community: incorporating physical information into predictive models often drastically reduces the data requirements. Leveraging MBDoE further increased the data efficiency. By design, the proposed data science framework is general in nature and can be easily extended to other experimental and additive manufacturing systems beyond AJP.

Hybrid Data-Driven Discovery of High-Performance Silver Selenide-Based Thermoelectric Composites

Optimizing material compositions often enhances thermoelectric performances. However, the large selection of possible base elements and dopants results in a vast composition design space that is too large to systematically search using solely domain knowledge. To address this challenge, a hybrid data-driven strategy that integrates Bayesian optimization (BO) and Gaussian process regression (GPR) is proposed to optimize the composition of five elements (Ag, Se, S, Cu, and Te) in AgSe-based thermoelectric materials. Data is collected from the literature to provide prior knowledge for the initial GPR model, which is updated by actively collected experimental data during the iteration between BO and experiments. Within seven iterations, the optimized AgSe-based materials prepared using a simple high-throughput ink mixing and blade coating method deliver a high power factor of 2100 µW m⁻¹ K⁻², which is a 75% improvement from the baseline composite (nominal composition of Ag₂Se₁). The success of this study provides opportunities to generalize the demonstrated active machine learning technique to accelerate the development and optimization of a wide range of material systems with reduced experimental trials.

Machine learning-assisted 3D printing of thermoelectric materials of ultrahigh performances at room temperature

Thermoelectric energy conversion is an attractive technology for generating electricity from waste heat and using electricity for solid-state cooling. However, conventional manufacturing processes for thermoelectric devices are costly and limited to simple device geometries. This work reports an extrusion printing method to fabricate high-performance thermoelectric materials with complex 3D architectures. By integrating high-throughput experimentation and Bayesian optimization (BO), our approach significantly accelerates the simultaneous search for the optimal ink formulation and printing parameters that deliver high thermoelectric performances while maintaining desired shape fidelity. A Gaussian process regression (GPR)-based machine learning model is employed to expeditiously predict thermoelectric power factor as a function of ink formulation and printing parameters. The printed bismuth antimony telluride (BiSbTe)-based thermoelectric materials under the optimized conditions exhibit an ultrahigh room temperature zT of 1.3, which is by far the highest in the printed thermoelectric materials. The machine learning-guided ink-based printing strategy can be easily generalized to a wide range of functional materials and devices for broad technological applications.

 

Bayesian Optimization of Low-Temperature Nonthermal Plasma Jet Sintering of Nanoinks

In response to the escalating demand for flexible devices in applications such as wearables, sensors, and touch panels, there is a need for innovative fabrication approaches for devices made from nanomaterial-based inks. Subsequent to ink deposition, a pivotal stage in device manufacturing typically involves high-temperature sintering, posing challenges for heat-sensitive substrates. Nonthermal plasma jet sintering utilizing an atmospheric pressure dielectric barrier discharge (DBD) plasma jet enables sintering at room temperature and standard pressure, facilitating the sintering of printed nanoparticle films without compromising substrate or film surface integrity. However, determining optimal plasma jet sintering conditions can be challenging due to multiple processing variables with intricate interrelationships. This work employed Bayesian optimization (BO) and machine learning (ML) to identify optimal values for seven primary plasma jet sintering variables. Optimization yielded a 99.2% increase in the measured electrical conductivity for plasma jet-sintered indium tin oxide (ITO) films after five rounds of experiments. Moreover, the optimal sintering conditions achieved an electrical conductivity that was 81.4% of conventional furnace sintering at 300 °C, but was three times faster and with a peak substrate temperature below 47 °C. This result demonstrates the prospect of applying BO to optimize processing techniques for emerging low-temperature requirements.

Gaussian Process Regression Machine Learning Models for Photonic Sintering

Novel solid-state thermoelectric (TE) materials have the potential to improve energy efficiency by converting waste heat into electricity. However, the performance of many state-of-the-art TE materials remains inadequate for adoption beyond niche applications. Current efforts to optimize photonic sintering, an important step in additive manufacturing of TE devices, rely on expert-driven trial-and-error search which is often extremely time-consuming and without the guarantee of improvement. Emerging Bayesian optimization frameworks offer a principled approach to intelligentially recommend optimized experimental conditions by balancing exploitation and exploration. In this paper, we develop a Gaussian Process Regression (GPR) machine learning model to predict the thermoelectric power factor of aerosol jet printed n-type Bi₂Te_2.7Se_0.3 TE films. We compare hyperparameter tuning methods and perform retrospective analysis to quantify the predictivity of GPR. Finally, we discuss the challenges and opportunities of adopting Bayesian optimization for photonic sintering and fabrication of high-performance TE devices.

Related Publications

[J1] Ke Wang, Alexander W. Dowling (2022). Bayesian optimization for chemical products and functional materials, Current Opinion in Chemical Engineering, 36, p. 100728

[J2] Mortaza Saeidi-Javash, Ke Wang, Minxiang Zeng, Tengfei Luo, Alexander Dowling, Yanliang Zhang (2022), Machine Learning-Assisted Ultrafast Flash Sintering of High-Performance and Wearable Silver-Selenide Thermoelectric Devices. (Energy&Environmental Science)

[J3] Ke Wang, Minxiang Zeng, Jialu Wang, Wenjie Shang, Yanliang Zhang, Tengfei Luo, Alexander Dowling (2022), When Physics-Informed Data Analytics Outperforms Black-box Machine Learning: A Case Study in Thickness Control for Additive Manufacturing. (Digital Chemical Engineering)

[J4] Wenjie Shang, Minxiang Zeng, Ali Tanvir, Ke Wang, Mortaza Saeidi-Javash, Alexander Dowling, Tengfei Luo, Yanliang Zhang. (2023) Hybrid Data-driven Discovery of High-performance Silver Selenide-based Thermoelectric Composites, Advanced Material.

[J5] Bridgette J. Befort, Alejandro Garciadiego, Jialu Wang, Ke Wang, Gabriela Franco, Edward J. Maginn, Alexander W. Dowling (2023). Data science for thermodynamic modeling: Case study for ionic liquid and hydrofluorocarbon refrigerant mixtures, Fluid Phase Equilibria.

[J6] Cheng, Z., Wang, K., Tanvir, A. M., Shang, W., Luo, T., Zhang, Y., A, Dowling. & Go, D. B. (2024). Bayesian Optimization of Low-Temperature Nonthermal Plasma Jet Sintering of Nanoinks. ACS Applied Materials & Interfaces, 16(35), 46897-46908.ials & Interfaces, 2024, 16(35): 46897-46908.

[J7] Tanvir, A. N. M., Bappy, M. O., Zeng, M., Shang, W., Wang, K., Song, K.,... & Zhang, Y. (2024). High-performance thermoelectric composites via scalable and low-cost ink processing. Energy & Environmental Science.

[J8] Song, K., Xu, G., Tanvir, A. N. M., Wang, K., Bappy, M. O., Yang, H., ... & Zhang, Y. (2024). Machine learning-assisted 3D printing of thermoelectric materials of ultrahigh performances at room temperature. Journal of Materials Chemistry A, 12(32), 21243-21251.

[C1] Ke Wang, Mortaza Saeidi-Javash, Minxiang Zeng, Zeyu Liu, Yanliang Zhang, Tengfei Luo, Alexander W. Dowling. Gaussian Process Regression Machine Learning Models for Photonic Sintering (2022). 14th International Symposium on Process Systems Engineering (PSE2021+). Ed. by Y. Yamashita, M. Kano.