The Inverse of Exact Renormalization Group Flows as Statistical Inference
0
🤯
Sign in to get full access
Overview
- The paper explores the connection between the Exact Renormalization Group (ERG) and Bayesian Statistical Inference through the lens of Optimal Transport and Dynamical Bayesian Inference.
- It demonstrates how the Dynamical Bayesian Inference equation is equivalent to a diffusion equation, which the authors dub "Bayesian Diffusion."
- By identifying the features of Bayesian Diffusion and mapping them to the ERG, the paper presents a new perspective on how renormalization can be understood as the inverse of statistical inference.
Plain English Explanation
The paper examines the relationship between two complex concepts: the Exact Renormalization Group (ERG) and Bayesian Statistical Inference. The ERG is a way of simplifying and understanding complex systems, while Bayesian Inference is a method of updating our beliefs about the world based on new information.
The researchers show that these two ideas are actually closely related, and can be understood through the lens of Optimal Transport and Dynamical Bayesian Inference. Dynamical Bayesian Inference is a way of encoding Bayesian Inference into a mathematical equation, and the researchers demonstrate that this equation is equivalent to a diffusion equation, which they call "Bayesian Diffusion."
By identifying the key features of Bayesian Diffusion and mapping them to the features of the ERG, the researchers are able to show how renormalization (the process of simplifying complex systems) can be understood as the inverse of statistical inference (the process of updating our beliefs based on new information). This provides a new way of thinking about these fundamental concepts in physics and mathematics.
Technical Explanation
The paper builds on the view of the Exact Renormalization Group (ERG) as an instantiation of Optimal Transport described by a functional convection-diffusion equation. The authors provide a new information-theoretic perspective for understanding the ERG through the intermediary of Bayesian Statistical Inference.
This connection is facilitated by the Dynamical Bayesian Inference scheme, which encodes Bayesian inference in the form of a one-parameter family of probability distributions solving an integro-differential equation derived from Bayes' law. The authors demonstrate how the Dynamical Bayesian Inference equation is, itself, equivalent to a diffusion equation which they dub "Bayesian Diffusion."
By identifying the features that define Bayesian Diffusion, and mapping them onto the features that define the ERG, the authors obtain a dictionary outlining how renormalization can be understood as the inverse of statistical inference.
Critical Analysis
The paper presents a novel and intriguing perspective on the relationship between the Exact Renormalization Group and Bayesian Statistical Inference. The authors' use of Optimal Transport and Dynamical Bayesian Inference as intermediaries provides a compelling framework for bridging these two complex concepts.
One potential limitation of the research is the lack of empirical validation or specific applications of the proposed dictionary mapping renormalization to statistical inference. While the theoretical connections are well-established, more work may be needed to demonstrate the practical implications and usefulness of this new viewpoint.
Additionally, the paper does not address potential challenges or caveats in applying this framework to real-world systems. Further research may be needed to understand the limitations of this approach and identify any potential pitfalls or areas for improvement.
Nevertheless, the paper's innovative perspective and the authors' rigorous analysis make it a thought-provoking contribution to the field. Readers are encouraged to think critically about the research and consider how it might inform their understanding of renormalization, Bayesian inference, and the broader connections between physics and information theory.
Conclusion
This paper presents a novel information-theoretic perspective on the Exact Renormalization Group (ERG) by establishing a connection to Bayesian Statistical Inference through the intermediary of Optimal Transport and Dynamical Bayesian Inference. By demonstrating the equivalence between the Dynamical Bayesian Inference equation and a diffusion equation, the authors develop a "Bayesian Diffusion" framework that allows them to map the features of the ERG to the inverse of statistical inference.
This work provides a fresh way of understanding the relationship between these fundamental concepts in physics and mathematics, with potentially far-reaching implications for fields such as complex systems, statistical mechanics, and machine learning. The paper's innovative approach and rigorous analysis make it a valuable contribution to the ongoing exploration of the deep connections between information, inference, and the nature of physical reality.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🤯
0
The Inverse of Exact Renormalization Group Flows as Statistical Inference
David S. Berman, Marc S. Klinger
We build on the view of the Exact Renormalization Group (ERG) as an instantiation of Optimal Transport described by a functional convection-diffusion equation. We provide a new information theoretic perspective for understanding the ERG through the intermediary of Bayesian Statistical Inference. This connection is facilitated by the Dynamical Bayesian Inference scheme, which encodes Bayesian inference in the form of a one parameter family of probability distributions solving an integro-differential equation derived from Bayes' law. In this note, we demonstrate how the Dynamical Bayesian Inference equation is, itself, equivalent to a diffusion equation which we dub Bayesian Diffusion. Identifying the features that define Bayesian Diffusion, and mapping them onto the features that define the ERG, we obtain a dictionary outlining how renormalization can be understood as the inverse of statistical inference.
Read more5/2/2024
0
Bayesian RG Flow in Neural Network Field Theories
Jessica N. Howard, Marc S. Klinger, Anindita Maiti, Alexander G. Stapleton
The Neural Network Field Theory correspondence (NNFT) is a mapping from neural network (NN) architectures into the space of statistical field theories (SFTs). The Bayesian renormalization group (BRG) is an information-theoretic coarse graining scheme that generalizes the principles of the Exact Renormalization Group (ERG) to arbitrarily parameterized probability distributions, including those of NNs. In BRG, coarse graining is performed in parameter space with respect to an information-theoretic distinguishability scale set by the Fisher information metric. In this paper, we unify NNFT and BRG to form a powerful new framework for exploring the space of NNs and SFTs, which we coin BRG-NNFT. With BRG-NNFT, NN training dynamics can be interpreted as inducing a flow in the space of SFTs from the information-theoretic `IR' $rightarrow$ `UV'. Conversely, applying an information-shell coarse graining to the trained network's parameters induces a flow in the space of SFTs from the information-theoretic `UV' $rightarrow$ `IR'. When the information-theoretic cutoff scale coincides with a standard momentum scale, BRG is equivalent to ERG. We demonstrate the BRG-NNFT correspondence on two analytically tractable examples. First, we construct BRG flows for trained, infinite-width NNs, of arbitrary depth, with generic activation functions. As a special case, we then restrict to architectures with a single infinitely-wide layer, scalar outputs, and generalized cos-net activations. In this case, we show that BRG coarse-graining corresponds exactly to the momentum-shell ERG flow of a free scalar SFT. Our analytic results are corroborated by a numerical experiment in which an ensemble of asymptotically wide NNs are trained and subsequently renormalized using an information-shell BRG scheme.
Read more5/29/2024
0
Wilsonian Renormalization of Neural Network Gaussian Processes
Jessica N. Howard, Ro Jefferson, Anindita Maiti, Zohar Ringel
Separating relevant and irrelevant information is key to any modeling process or scientific inquiry. Theoretical physics offers a powerful tool for achieving this in the form of the renormalization group (RG). Here we demonstrate a practical approach to performing Wilsonian RG in the context of Gaussian Process (GP) Regression. We systematically integrate out the unlearnable modes of the GP kernel, thereby obtaining an RG flow of the GP in which the data sets the IR scale. In simple cases, this results in a universal flow of the ridge parameter, which becomes input-dependent in the richer scenario in which non-Gaussianities are included. In addition to being analytically tractable, this approach goes beyond structural analogies between RG and neural networks by providing a natural connection between RG flow and learnable vs. unlearnable modes. Studying such flows may improve our understanding of feature learning in deep neural networks, and enable us to identify potential universality classes in these models.
Read more8/15/2024
0
Inflationary Flows: Calibrated Bayesian Inference with Diffusion-Based Models
Daniela de Albuquerque, John Pearson
Beyond estimating parameters of interest from data, one of the key goals of statistical inference is to properly quantify uncertainty in these estimates. In Bayesian inference, this uncertainty is provided by the posterior distribution, the computation of which typically involves an intractable high-dimensional integral. Among available approximation methods, sampling-based approaches come with strong theoretical guarantees but scale poorly to large problems, while variational approaches scale well but offer few theoretical guarantees. In particular, variational methods are known to produce overconfident estimates of posterior uncertainty and are typically non-identifiable, with many latent variable configurations generating equivalent predictions. Here, we address these challenges by showing how diffusion-based models (DBMs), which have recently produced state-of-the-art performance in generative modeling tasks, can be repurposed for performing calibrated, identifiable Bayesian inference. By exploiting a previously established connection between the stochastic and probability flow ordinary differential equations (pfODEs) underlying DBMs, we derive a class of models, inflationary flows, that uniquely and deterministically map high-dimensional data to a lower-dimensional Gaussian distribution via ODE integration. This map is both invertible and neighborhood-preserving, with controllable numerical error, with the result that uncertainties in the data are correctly propagated to the latent space. We demonstrate how such maps can be learned via standard DBM training using a novel noise schedule and are effective at both preserving and reducing intrinsic data dimensionality. The result is a class of highly expressive generative models, uniquely defined on a low-dimensional latent space, that afford principled Bayesian inference.
Read more8/22/2024