Contribute to balance, wire in accordance: Emergence of backpropagation from a simple, bio-plausible neuroplasticity rule

Overview

• The paper introduces a novel neuroplasticity rule that could provide a biologically plausible mechanism for implementing backpropagation (BP) in the brain.
• The proposed rule is similar to the classical Hebbian rule but operates on three different timescales: neural firing, retrograde signaling, and neural plasticity.
• The authors demonstrate mathematically that their learning rule precisely replicates BP in layered neural networks without any approximations.
• Simulations on artificial neural networks reveal that this rule induces varying community structures in networks, depending on the learning rate.

Plain English Explanation

The paper discusses a potential way that the brain could implement a technique called backpropagation, which is widely used in machine learning but has been difficult to explain in biological terms. The authors propose a new rule for how connections between neurons could be strengthened or weakened over time.

This rule is similar to the Hebbian learning idea, where connections between neurons that fire together tend to get stronger. However, the new rule also takes into account a neuron's "credit" - how much it contributes to maintaining the balance between excitatory and inhibitory inputs in the neurons it connects to.

After the network's firing rates reach a stable state, neurons receive credits based on this balance, and the connections from the neurons that significantly contribute to this balance are then strengthened over a slower timescale. The authors show that this simple rule can actually replicate the effects of backpropagation, a powerful machine learning technique, without requiring the brain to do the complex computations that backpropagation normally involves.

Simulations also suggest that this rule can lead to the formation of different community structures in neural networks, depending on the learning rate. Overall, this provides a biologically plausible way to understand how the brain could implement something akin to backpropagation, with testable predictions that could be evaluated through experiments.

Technical Explanation

The paper introduces a novel neuroplasticity rule that offers a potential mechanism for implementing backpropagation (BP) in the brain. The rule is based on the core principles of maintaining the balance of excitatory and inhibitory inputs, as well as on retrograde signaling, and operates over three progressively slower timescales: neural firing, retrograde signaling, and neural plasticity.

The authors hypothesize that each neuron possesses an internal state, termed "credit," in addition to its firing rate. After the network reaches equilibrium in firing rates, neurons receive credits based on their contribution to the E-I balance of postsynaptic neurons through retrograde signaling. As the network's credit distribution stabilizes, connections from those presynaptic neurons are strengthened that significantly contribute to the balance of postsynaptic neurons.

The authors demonstrate mathematically that their learning rule precisely replicates BP in layered neural networks without any approximations. Simulations on artificial neural networks reveal that this rule induces varying community structures in networks, depending on the learning rate.

Critical Analysis

The authors present a novel and biologically plausible mechanism for implementing backpropagation in the brain, which is a significant contribution to the field. The use of retrograde signaling and the concept of neuronal credit are intriguing and provide a potential explanation for how the brain could achieve something akin to backpropagation without the need for symmetrical connections or distinct forward and backward phases of computation.

However, the authors acknowledge that their framework relies on several assumptions that would need to be validated through biological experiments. For example, the existence of a "credit" signal and the specific mechanisms of retrograde signaling and their integration with neural activity and plasticity processes would need to be empirically demonstrated.

Additionally, while the simulations on artificial neural networks are informative, it would be valuable to see the model's performance on more realistic neural network architectures and tasks to better understand its biological relevance and potential applications. Further research could also explore the model's robustness to factors such as noise, sparsity, and heterogeneity in neuronal properties.

Overall, the authors have presented a compelling theoretical framework that improves upon existing ideas like Hebbian learning and contrastive learning to offer a biologically plausible implementation of backpropagation. The testable predictions and the potential for experimental validation make this a promising area for future research on the transition from local to global order in neural networks.

Conclusion

The paper presents a novel neuroplasticity rule that offers a potential mechanism for implementing backpropagation, a widely used machine learning technique, in the brain. The rule is based on the principles of maintaining the balance of excitatory and inhibitory inputs and retrograde signaling, and operates on multiple timescales.

The authors demonstrate that this rule can precisely replicate backpropagation in layered neural networks without approximations, and simulations suggest it can induce varying community structures in artificial neural networks. While the framework relies on several testable assumptions, it provides a biologically plausible way to understand how the brain could implement something akin to backpropagation, with potential implications for our understanding of neural information processing and learning.