Uncertainty Quantification in Multivariable Regression for Material Property Prediction with Bayesian Neural Networks
2311.02495
0
0
↗️
Abstract
With the increased use of data-driven approaches and machine learning-based methods in material science, the importance of reliable uncertainty quantification (UQ) of the predicted variables for informed decision-making cannot be overstated. UQ in material property prediction poses unique challenges, including the multi-scale and multi-physics nature of advanced materials, intricate interactions between numerous factors, limited availability of large curated datasets for model training, etc. Recently, Bayesian Neural Networks (BNNs) have emerged as a promising approach for UQ, offering a probabilistic framework for capturing uncertainties within neural networks. In this work, we introduce an approach for UQ within physics-informed BNNs, which integrates knowledge from governing laws in material modeling to guide the models toward physically consistent predictions. To evaluate the effectiveness of this approach, we present case studies for predicting the creep rupture life of steel alloys. Experimental validation with three datasets of collected measurements from creep tests demonstrates the ability of BNNs to produce accurate point and uncertainty estimates that are competitive or exceed the performance of the conventional method of Gaussian Process Regression. Similarly, we evaluated the suitability of BNNs for UQ in an active learning application and reported competitive performance. The most promising framework for creep life prediction is BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs. The codes are available at: https://github.com/avakanski/Creep-uncertainty-quantification.
Create account to get full access
Overview
- Importance of reliable uncertainty quantification (UQ) in material science and machine learning-based methods
- Unique challenges in UQ for material property prediction
- Bayesian Neural Networks (BNNs) as a promising approach for UQ
- Integration of physical knowledge to guide BNNs toward consistent predictions
- Evaluation of BNNs for creep rupture life prediction in steel alloys
Plain English Explanation
Predicting the properties of materials is crucial in many industries, but it's a complex task with a lot of uncertainty. As machine learning becomes more widely used in materials science, having reliable ways to quantify this uncertainty is essential for making informed decisions. Predicting Safety Misbehaviours in Autonomous Driving Systems Using Uncertainty Quantification (UQ) techniques can help address the unique challenges in this field, like the multi-scale and multi-physics nature of advanced materials and the limited availability of large datasets.
One promising approach is Bayesian Neural Networks (BNNs), which provide a probabilistic framework for capturing uncertainties within neural networks. In this work, the researchers combined BNNs with physical knowledge about material behavior to guide the models towards more accurate and consistent predictions. They tested this approach on predicting the creep rupture life of steel alloys, which is an important property for many industrial applications.
The researchers found that BNNs were able to produce accurate point estimates and reliable uncertainty estimates, performing as well as or better than a conventional method called Gaussian Process Regression. They also evaluated BNNs for active learning, where the model can request additional data to improve its performance, and found the BNNs performed competitively. The most promising approach was using Markov Chain Monte Carlo to approximate the BNN's posterior distribution, as it provided more reliable results than other BNN variants or related neural networks with probabilistic outputs.
Technical Explanation
The researchers introduced an approach for UQ within physics-informed BNNs, which integrates knowledge from governing laws in material modeling to guide the models toward physically consistent predictions. To evaluate the effectiveness of this approach, they presented case studies for predicting the creep rupture life of steel alloys.
Experimental validation with three datasets of collected measurements from creep tests demonstrated the ability of BNNs to produce accurate point and uncertainty estimates that are competitive or exceed the performance of the conventional method of Gaussian Process Regression. The researchers also evaluated the suitability of BNNs for UQ in an active learning application and reported competitive performance.
The most promising framework for creep life prediction was BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs.
Critical Analysis
The paper addresses an important challenge in materials science by integrating physical knowledge into BNNs to improve the reliability of uncertainty quantification. The experimental validation with real-world datasets and the comparison to a conventional method, Gaussian Process Regression, provide a robust evaluation of the proposed approach.
However, the paper does not discuss the computational complexity or training time of the different BNN variants, which could be a practical consideration for real-world applications. Additionally, the paper does not explore the sensitivity of the BNN performance to the amount and quality of the available dataset, which is a common concern in Epistemic Uncertainty Quantification in Pre-Trained Neural Networks.
Further research could investigate the generalization of the physics-informed BNN approach to other material properties and explore strategies for efficiently incorporating domain knowledge into the model architecture and training process.
Conclusion
This work demonstrates the potential of Bayesian Neural Networks for reliable uncertainty quantification in material property prediction, a critical task in materials science. By integrating physical knowledge into the BNN framework, the researchers were able to produce accurate point estimates and reliable uncertainty estimates for creep rupture life, outperforming a conventional method.
The findings contribute to the growing body of research on Uncertainty Quantification in Graph Learning and highlight the value of incorporating domain-specific knowledge into machine learning models for materials science applications. The open-source code provided by the authors will likely be a valuable resource for further exploration and development in this area.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
Bayesian neural networks for predicting uncertainty in full-field material response
George D. Pasparakis, Lori Graham-Brady, Michael D. Shields
0
0
Stress and material deformation field predictions are among the most important tasks in computational mechanics. These predictions are typically made by solving the governing equations of continuum mechanics using finite element analysis, which can become computationally prohibitive considering complex microstructures and material behaviors. Machine learning (ML) methods offer potentially cost effective surrogates for these applications. However, existing ML surrogates are either limited to low-dimensional problems and/or do not provide uncertainty estimates in the predictions. This work proposes an ML surrogate framework for stress field prediction and uncertainty quantification for diverse materials microstructures. A modified Bayesian U-net architecture is employed to provide a data-driven image-to-image mapping from initial microstructure to stress field with prediction (epistemic) uncertainty estimates. The Bayesian posterior distributions for the U-net parameters are estimated using three state-of-the-art inference algorithms: the posterior sampling-based Hamiltonian Monte Carlo method and two variational approaches, the Monte-Carlo Dropout method and the Bayes by Backprop algorithm. A systematic comparison of the predictive accuracy and uncertainty estimates for these methods is performed for a fiber reinforced composite material and polycrystalline microstructure application. It is shown that the proposed methods yield predictions of high accuracy compared to the FEA solution, while uncertainty estimates depend on the inference approach. Generally, the Hamiltonian Monte Carlo and Bayes by Backprop methods provide consistent uncertainty estimates. Uncertainty estimates from Monte Carlo Dropout, on the other hand, are more difficult to interpret and depend strongly on the method's design.
6/24/2024
🧠
Uncertainty Quantification for Molecular Property Predictions with Graph Neural Architecture Search
Shengli Jiang, Shiyi Qin, Reid C. Van Lehn, Prasanna Balaprakash, Victor M. Zavala
0
0
Graph Neural Networks (GNNs) have emerged as a prominent class of data-driven methods for molecular property prediction. However, a key limitation of typical GNN models is their inability to quantify uncertainties in the predictions. This capability is crucial for ensuring the trustworthy use and deployment of models in downstream tasks. To that end, we introduce AutoGNNUQ, an automated uncertainty quantification (UQ) approach for molecular property prediction. AutoGNNUQ leverages architecture search to generate an ensemble of high-performing GNNs, enabling the estimation of predictive uncertainties. Our approach employs variance decomposition to separate data (aleatoric) and model (epistemic) uncertainties, providing valuable insights for reducing them. In our computational experiments, we demonstrate that AutoGNNUQ outperforms existing UQ methods in terms of both prediction accuracy and UQ performance on multiple benchmark datasets. Additionally, we utilize t-SNE visualization to explore correlations between molecular features and uncertainty, offering insight for dataset improvement. AutoGNNUQ has broad applicability in domains such as drug discovery and materials science, where accurate uncertainty quantification is crucial for decision-making.
7/2/2024
🤿
A Comprehensive Survey on Uncertainty Quantification for Deep Learning
Wenchong He, Zhe Jiang
0
0
Deep neural networks (DNNs) have achieved tremendous success in making accurate predictions for computer vision, natural language processing, as well as science and engineering domains. However, it is also well-recognized that DNNs sometimes make unexpected, incorrect, but overconfident predictions. This can cause serious consequences in high-stake applications, such as autonomous driving, medical diagnosis, and disaster response. Uncertainty quantification (UQ) aims to estimate the confidence of DNN predictions beyond prediction accuracy. In recent years, many UQ methods have been developed for DNNs. It is of great practical value to systematically categorize these UQ methods and compare their advantages and disadvantages. However, existing surveys mostly focus on categorizing UQ methodologies from a neural network architecture perspective or a Bayesian perspective and ignore the source of uncertainty that each methodology can incorporate, making it difficult to select an appropriate UQ method in practice. To fill the gap, this paper presents a systematic taxonomy of UQ methods for DNNs based on the types of uncertainty sources (data uncertainty versus model uncertainty). We summarize the advantages and disadvantages of methods in each category. We show how our taxonomy of UQ methodologies can potentially help guide the choice of UQ method in different machine learning problems (e.g., active learning, robustness, and reinforcement learning). We also identify current research gaps and propose several future research directions.
4/11/2024
🤿
Uncertainty Quantification for Deep Learning
Peter Jan van Leeuwen, J. Christine Chiu, C. Kevin Yang
0
0
A complete and statistically consistent uncertainty quantification for deep learning is provided, including the sources of uncertainty arising from (1) the new input data, (2) the training and testing data (3) the weight vectors of the neural network, and (4) the neural network because it is not a perfect predictor. Using Bayes Theorem and conditional probability densities, we demonstrate how each uncertainty source can be systematically quantified. We also introduce a fast and practical way to incorporate and combine all sources of errors for the first time. For illustration, the new method is applied to quantify errors in cloud autoconversion rates, predicted from an artificial neural network that was trained by aircraft cloud probe measurements in the Azores and the stochastic collection equation formulated as a two-moment bin model. For this specific example, the output uncertainty arising from uncertainty in the training and testing data is dominant, followed by uncertainty in the input data, in the trained neural network, and uncertainty in the weights. We discuss the usefulness of the methodology for machine learning practice, and how, through inclusion of uncertainty in the training data, the new methodology is less sensitive to input data that falls outside of the training data set.
6/3/2024