Data-Driven Discovery of PDEs via the Adjoint Method
2401.17177
0
0
⛏️
Abstract
In this work, we present an adjoint-based method for discovering the underlying governing partial differential equations (PDEs) given data. The idea is to consider a parameterized PDE in a general form and formulate a PDE-constrained optimization problem aimed at minimizing the error of the PDE solution from data. Using variational calculus, we obtain an evolution equation for the Lagrange multipliers (adjoint equations) allowing us to compute the gradient of the objective function with respect to the parameters of PDEs given data in a straightforward manner. In particular, we consider a family of parameterized PDEs encompassing linear, nonlinear, and spatial derivative candidate terms, and elegantly derive the corresponding adjoint equations. We show the efficacy of the proposed approach in identifying the form of the PDE up to machine accuracy, enabling the accurate discovery of PDEs from data. We also compare its performance with the famous PDE Functional Identification of Nonlinear Dynamics method known as PDE-FIND (Rudy et al., 2017), on both smooth and noisy data sets. Even though the proposed adjoint method relies on forward/backward solvers, it outperforms PDE-FIND for large data sets thanks to the analytic expressions for gradients of the cost function with respect to each PDE parameter.
Create account to get full access
Overview
- Presents an adjoint-based method for discovering underlying partial differential equations (PDEs) from data
- Formulates a PDE-constrained optimization problem to minimize the error between PDE solution and data
- Derives adjoint equations to efficiently compute gradients of the objective function with respect to PDE parameters
- Shows the method can identify the correct PDE form to machine accuracy, outperforming existing approaches
Plain English Explanation
This research proposes a new way to uncover the underlying partial differential equations that govern a dynamic system, given only observational data about that system. The key idea is to start with a very general family of candidate PDEs, and then use an optimization process to find the specific PDE parameters that best match the observed data.
The optimization problem is set up such that the PDE solution has to closely match the data. To solve this problem efficiently, the researchers derive "adjoint equations" - a type of "backward" equation that allows the gradients of the objective function to be computed analytically with respect to each PDE parameter. This makes the optimization much faster and more accurate than previous methods like PDE-FIND, especially for large datasets.
The researchers demonstrate that their adjoint-based method can identify the correct PDE form to machine precision, outperforming other approaches. This is a significant advance, as being able to automatically discover the governing equations from data has many important applications in science and engineering, such as discovering novel integrable systems.
Technical Explanation
The key technical elements of this work are:
-
Formulating a PDE-constrained optimization problem: The researchers start with a general parameterized PDE form, and define an objective function that measures the error between the PDE solution and the observed data.
-
Deriving the adjoint equations: Using variational calculus, they derive the "adjoint equations" - a set of evolution equations for Lagrange multipliers that allow the gradients of the objective function to be computed efficiently with respect to the PDE parameters.
-
Experiment design: The researchers consider a family of PDEs that encompass linear, nonlinear, and spatial derivative terms. They test their adjoint-based method on both smooth and noisy datasets, and compare its performance to the PDE-FIND algorithm.
-
Results: The experiments demonstrate that the proposed adjoint-based method can identify the correct PDE form to machine accuracy, outperforming PDE-FIND, especially for large datasets. This is due to the analytic expressions for the gradients provided by the adjoint equations.
Critical Analysis
The paper provides a thorough technical description and evaluation of the proposed adjoint-based PDE discovery method. However, a few potential limitations and areas for further research are worth noting:
-
The method assumes the PDE form is known up to a set of parameters, which may not always be the case in practice. Extending the approach to discover the PDE structure from a more general class of candidate equations could be valuable.
-
The experiments are limited to relatively low-dimensional PDE systems. Scaling the method to high-dimensional PDEs with thousands of parameters, as in the OptPDE work, would be an important test of its practical applicability.
-
The paper does not discuss the sensitivity of the method to model misspecification or the impact of normalization/scaling choices on the optimization landscape and convergence.
Overall, this is a promising approach that advances the state-of-the-art in data-driven PDE discovery. Further research to address the above limitations could lead to even more robust and widely applicable PDE discovery tools.
Conclusion
This work presents a new adjoint-based method for discovering the underlying partial differential equations that govern a dynamic system, given only observational data about that system. By formulating a PDE-constrained optimization problem and deriving efficient adjoint equations, the researchers demonstrate that their approach can identify the correct PDE form to machine accuracy, outperforming existing methods.
The ability to automatically uncover the governing equations from data has many important applications in science and engineering, such as discovering novel integrable systems. While the current method has some limitations, this research represents a significant step forward in the field of data-driven PDE discovery.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
💬
Automating the Discovery of Partial Differential Equations in Dynamical Systems
Weizhen Li, Rui Carvalho
0
0
Identifying partial differential equations (PDEs) from data is crucial for understanding the governing mechanisms of natural phenomena, yet it remains a challenging task. We present an extension to the ARGOS framework, ARGOS-RAL, which leverages sparse regression with the recurrent adaptive lasso to identify PDEs from limited prior knowledge automatically. Our method automates calculating partial derivatives, constructing a candidate library, and estimating a sparse model. We rigorously evaluate the performance of ARGOS-RAL in identifying canonical PDEs under various noise levels and sample sizes, demonstrating its robustness in handling noisy and non-uniformly distributed data. We also test the algorithm's performance on datasets consisting solely of random noise to simulate scenarios with severely compromised data quality. Our results show that ARGOS-RAL effectively and reliably identifies the underlying PDEs from data, outperforming the sequential threshold ridge regression method in most cases. We highlight the potential of combining statistical methods, machine learning, and dynamical systems theory to automatically discover governing equations from collected data, streamlining the scientific modeling process.
5/3/2024
📊
Constrained or Unconstrained? Neural-Network-Based Equation Discovery from Data
Grant Norman, Jacqueline Wentz, Hemanth Kolla, Kurt Maute, Alireza Doostan
0
0
Throughout many fields, practitioners often rely on differential equations to model systems. Yet, for many applications, the theoretical derivation of such equations and/or accurate resolution of their solutions may be intractable. Instead, recently developed methods, including those based on parameter estimation, operator subset selection, and neural networks, allow for the data-driven discovery of both ordinary and partial differential equations (PDEs), on a spectrum of interpretability. The success of these strategies is often contingent upon the correct identification of representative equations from noisy observations of state variables and, as importantly and intertwined with that, the mathematical strategies utilized to enforce those equations. Specifically, the latter has been commonly addressed via unconstrained optimization strategies. Representing the PDE as a neural network, we propose to discover the PDE by solving a constrained optimization problem and using an intermediate state representation similar to a Physics-Informed Neural Network (PINN). The objective function of this constrained optimization problem promotes matching the data, while the constraints require that the PDE is satisfied at several spatial collocation points. We present a penalty method and a widely used trust-region barrier method to solve this constrained optimization problem, and we compare these methods on numerical examples. Our results on the Burgers' and the Korteweg-De Vreis equations demonstrate that the latter constrained method outperforms the penalty method, particularly for higher noise levels or fewer collocation points. For both methods, we solve these discovered neural network PDEs with classical methods, such as finite difference methods, as opposed to PINNs-type methods relying on automatic differentiation. We briefly highlight other small, yet crucial, implementation details.
6/6/2024
📊
Physics-constrained robust learning of open-form partial differential equations from limited and noisy data
Mengge Du, Yuntian Chen, Longfeng Nie, Siyu Lou, Dongxiao Zhang
0
0
Unveiling the underlying governing equations of nonlinear dynamic systems remains a significant challenge. Insufficient prior knowledge hinders the determination of an accurate candidate library, while noisy observations lead to imprecise evaluations, which in turn result in redundant function terms or erroneous equations. This study proposes a framework to robustly uncover open-form partial differential equations (PDEs) from limited and noisy data. The framework operates through two alternating update processes: discovering and embedding. The discovering phase employs symbolic representation and a novel reinforcement learning (RL)-guided hybrid PDE generator to efficiently produce diverse open-form PDEs with tree structures. A neural network-based predictive model fits the system response and serves as the reward evaluator for the generated PDEs. PDEs with higher rewards are utilized to iteratively optimize the generator via the RL strategy and the best-performing PDE is selected by a parameter-free stability metric. The embedding phase integrates the initially identified PDE from the discovering process as a physical constraint into the predictive model for robust training. The traversal of PDE trees automates the construction of the computational graph and the embedding process without human intervention. Numerical experiments demonstrate our framework's capability to uncover governing equations from nonlinear dynamic systems with limited and highly noisy data and outperform other physics-informed neural network-based discovery methods. This work opens new potential for exploring real-world systems with limited understanding.
4/30/2024
🌀
One-shot learning for solution operators of partial differential equations
Anran Jiao, Haiyang He, Rishikesh Ranade, Jay Pathak, Lu Lu
0
0
Learning and solving governing equations of a physical system, represented by partial differential equations (PDEs), from data is a central challenge in a variety of areas of science and engineering. Traditional numerical methods for solving PDEs can be computationally expensive for complex systems and require the complete PDEs of the physical system. On the other hand, current data-driven machine learning methods require a large amount of data to learn a surrogate model of the PDE solution operator, which could be impractical. Here, we propose the first solution operator learning method that only requires one PDE solution, i.e., one-shot learning. By leveraging the principle of locality of PDEs, we consider small local domains instead of the entire computational domain and define a local solution operator. The local solution operator is then trained using a neural network, and utilized to predict the solution of a new input function via mesh-based fixed-point iteration (FPI), meshfree local-solution-operator informed neural network (LOINN) or local-solution-operator informed neural network with correction (cLOINN). We test our method on diverse PDEs, including linear or nonlinear PDEs, PDEs defined on complex geometries, and PDE systems, demonstrating the effectiveness and generalization capabilities of our method across these varied scenarios.
6/10/2024