Measurement-driven neural-network training for integrated magnetic tunnel junction arrays
0
Sign in to get full access
Overview
- Demonstrates a robust training method for neuromorphic hardware with high defect rates
- Focuses on magnetic tunnel junction (MTJ) devices integrated with CMOS circuitry
- Evaluates the performance of the proposed training approach on a large-scale MTJ array
Plain English Explanation
The research paper describes an experimental demonstration of a robust training method for strongly defective neuromorphic hardware. Neuromorphic hardware, which is inspired by the human brain, can be prone to high defect rates due to the complexity of the underlying devices. The researchers address this challenge by developing a training approach that can effectively handle these defects.
The focus of the study is on magnetic tunnel junction (MTJ) devices integrated with complementary metal-oxide-semiconductor (CMOS) circuitry. MTJs are a type of memory device that can be used to build neuromorphic systems. The researchers evaluate the performance of their proposed training method on a large-scale array of 20,000 MTJ devices.
Technical Explanation
The experimental setup involves the design and testing of 20,000 MTJs integrated with CMOS circuitry. The researchers develop a training approach that can effectively handle the high defect rates inherent in this type of neuromorphic hardware.
The training method leverages the exploration of novel neuromorphic methodologies and materials to achieve robust performance in the presence of defects. The researchers evaluate the effectiveness of their approach on the large-scale MTJ array, demonstrating its ability to mitigate the impact of defects and maintain high computational capabilities.
Critical Analysis
The paper provides a detailed and rigorous experimental demonstration of a robust training method for strongly defective neuromorphic hardware. The researchers have addressed a significant challenge in the field of neuromorphic computing, which is the need to develop techniques that can handle high defect rates in the underlying hardware.
One potential limitation of the study is the specific focus on MTJ devices, which may limit the generalizability of the findings to other types of neuromorphic hardware. Additionally, the paper does not discuss the scalability of the proposed training method to even larger-scale systems or its performance on more complex computational tasks.
Further research could explore the applicability of the training approach to a wider range of neuromorphic architectures and evaluate its long-term reliability and performance under real-world conditions.
Conclusion
The research paper presents an experimental demonstration of a robust training method for strongly defective neuromorphic hardware. The proposed approach effectively mitigates the impact of high defect rates in MTJ-based neuromorphic systems, enabling reliable and high-performing computational capabilities. This work represents an important step forward in addressing a key challenge in the field of neuromorphic computing and could have significant implications for the development of future brain-inspired hardware systems.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
Measurement-driven neural-network training for integrated magnetic tunnel junction arrays
William A. Borders, Advait Madhavan, Matthew W. Daniels, Vasileia Georgiou, Martin Lueker-Boden, Tiffany S. Santos, Patrick M. Braganca, Mark D. Stiles, Jabez J. McClelland, Brian D. Hoskins
The increasing scale of neural networks needed to support more complex applications has led to an increasing requirement for area- and energy-efficient hardware. One route to meeting the budget for these applications is to circumvent the von Neumann bottleneck by performing computation in or near memory. An inevitability of transferring neural networks onto hardware is that non-idealities such as device-to-device variations or poor device yield impact performance. Methods such as hardware-aware training, where substrate non-idealities are incorporated during network training, are one way to recover performance at the cost of solution generality. In this work, we demonstrate inference on hardware neural networks consisting of 20,000 magnetic tunnel junction arrays integrated on a complementary metal-oxide-semiconductor chips that closely resembles market-ready spin transfer-torque magnetoresistive random access memory technology. Using 36 dies, each containing a crossbar array with its own non-idealities, we show that even a small number of defects in physically mapped networks significantly degrades the performance of networks trained without defects and show that, at the cost of generality, hardware-aware training accounting for specific defects on each die can recover to comparable performance with ideal networks. We then demonstrate a robust training method that extends hardware-aware training to statistics-aware training, producing network weights that perform well on most defective dies regardless of their specific defect locations. When evaluated on the 36 physical dies, statistics-aware trained solutions can achieve a mean misclassification error on the MNIST dataset that differs from the software-baseline by only 2 %. This statistics-aware training method could be generalized to networks with many layers that are mapped to hardware suited for industry-ready applications.
Read more5/15/2024
👀
0
A design of magnetic tunnel junctions for the deployment of neuromorphic hardware for edge computing
Davi Rodrigues, Eleonora Raimondo, Riccardo Tomasello, Mario Carpentieri, Giovanni Finocchio
The electrically readable complex dynamics of robust and scalable magnetic tunnel junctions (MTJs) offer promising opportunities for advancing neuromorphic computing. In this work, we present an MTJ design with a free layer and two polarizers capable of computing the sigmoidal activation function and its gradient at the device level. This design enables both feedforward and backpropagation computations within a single device, extending neuromorphic computing frameworks previously explored in the literature by introducing the ability to perform backpropagation directly in hardware. Our algorithm implementation reveals two key findings: (i) the small discrepancies between the MTJ-generated curves and the exact software-generated curves have a negligible impact on the performance of the backpropagation algorithm, (ii) the device implementation is highly robust to inter-device variation and noise, and (iii) the proposed method effectively supports transfer learning and knowledge distillation. To demonstrate this, we evaluated the performance of an edge computing network using weights from a software-trained model implemented with our MTJ design. The results show a minimal loss of accuracy of only 0.1% for the Fashion MNIST dataset and 2% for the CIFAR-100 dataset compared to the original software implementation. These results highlight the potential of our MTJ design for compact, hardware-based neural networks in edge computing applications, particularly for transfer learning.
Read more9/5/2024
🧠
0
On-Chip Learning with Memristor-Based Neural Networks: Assessing Accuracy and Efficiency Under Device Variations, Conductance Errors, and Input Noise
M. Reza Eslami, Dhiman Biswas, Soheib Takhtardeshir, Sarah S. Sharif, Yaser M. Banad
This paper presents a memristor-based compute-in-memory hardware accelerator for on-chip training and inference, focusing on its accuracy and efficiency against device variations, conductance errors, and input noise. Utilizing realistic SPICE models of commercially available silver-based metal self-directed channel (M-SDC) memristors, the study incorporates inherent device non-idealities into the circuit simulations. The hardware, consisting of 30 memristors and 4 neurons, utilizes three different M-SDC structures with tungsten, chromium, and carbon media to perform binary image classification tasks. An on-chip training algorithm precisely tunes memristor conductance to achieve target weights. Results show that incorporating moderate noise (<15%) during training enhances robustness to device variations and noisy input data, achieving up to 97% accuracy despite conductance variations and input noises. The network tolerates a 10% conductance error without significant accuracy loss. Notably, omitting the initial memristor reset pulse during training considerably reduces training time and energy consumption. The hardware designed with chromium-based memristors exhibits superior performance, achieving a training time of 2.4 seconds and an energy consumption of 18.9 mJ. This research provides insights for developing robust and energy-efficient memristor-based neural networks for on-chip learning in edge applications.
Read more8/28/2024
0
Domain wall and Magnetic Tunnel Junction Hybrid for on-chip Learning in UNet architecture
Venkatesh Vadde, Bhaskaran Muralidharan, Abhishek Sharma
We present spintronic devices based hardware implementation of UNet for segmentation tasks. Our approach involves designing hardware for convolution, deconvolution, rectified activation function (ReLU), and max pooling layers of the UNet architecture. We designed the convolution and deconvolution layers of the network using the synaptic behavior of the domain wall MTJ. We also construct the ReLU and max pooling functions of the network utilizing the spin hall driven orthogonal current injected MTJ. To incorporate the diverse physics of spin-transport, magnetization dynamics, and CMOS elements in our UNet design, we employ a hybrid simulation setup that couples micromagnetic simulation, non-equilibrium Green's function, SPICE simulation along with network implementation. We evaluate our UNet design on the CamVid dataset and achieve segmentation accuracies of 83.71$%$ on test data, on par with the software implementation with 821mJ of energy consumption for on-chip training over 150 epochs. We further demonstrate nearly one order $(10times)$ improvement in the energy requirement of the network using unstable ferromagnet ($Delta$=4.58) over the stable ferromagnet ($Delta$=45) based ReLU and max pooling functions while maintaining the similar accuracy. The hybrid architecture comprising domain wall MTJ and unstable FM-based MTJ leads to an on-chip energy consumption of 85.79mJ during training, with a testing energy cost of 1.55 $mu J$.
Read more7/12/2024