Prospects for non-linear memristors as so-far missing core hardware element for transferless data computing and storage
2403.20051
0
0
📊
Abstract
We like and need Information and Communications Technologies (ICT) for data processing. This is measureable in the exponential growth of data processed by ICT, e.g. ICT for cryptocurrency mining and search engines. So far, the energy demand for computing technology has increased by a factor of 1.38 every ten years due to the exponentially increasing use of ICT systems as computing devices. The energy consumption of ICT systems is expected to rise from 1500 TWh (8% of global electricity consumption) in 2010 to 5700 TWh (14% of global electricity consumption) in 2030. A large part of this energy is required for the continuous data transfer between the separated memory and processor units which constitute the main components of ICT computing devices in von-Neumann architecture. This at the same time massively slows down the computing power of ICT systems in the von-Neumann architecture. In addition, due to the increasing complexity of AI compute algorithms, since 2010 the AI training compute time demand for computing technology increases tenfold every year, for example in the period from 2010 to 2020 from 1x10^{-6} to 1x10^{+4} Petaflops/Day. It has been theoretically predicted that ICT systems in the neuromorphic computer architecture will circumvent all of this through the use of merged memory and processor units. However, the core hardware element for this has not yet been realized so far. In this work we discuss the prespectives for non-linear resistive switches as the core hardware element for merged memory and processor units in neuromorphic computers.
Get summaries of the top AI research delivered straight to your inbox:
Overview
- Exponential growth in data processing by information and communications technologies (ICT) has led to a significant increase in energy consumption
- The von-Neumann architecture, which separates memory and processor units, results in slow computing power and high energy demands
- Neuromorphic computing using merged memory and processor units could address these issues, but the core hardware element has not yet been realized
- This paper discusses the potential of non-linear resistive switches as the core hardware element for neuromorphic computers
Plain English Explanation
The world has become increasingly reliant on information and communications technologies (ICT). Things like cryptocurrency mining, search engines, and other digital technologies process vast amounts of data. This data processing has led to a rapid increase in the energy required by ICT systems.
Over the past decade, the energy consumption of ICT has grown by about 40% every ten years. It's estimated to rise from 8% of global electricity use in 2010 to 14% by 2030. A major reason for this high energy use is the way traditional computer architectures, based on the von-Neumann model, separate memory and processing units. This separation results in a lot of back-and-forth data transfer, which slows down computing power.
Additionally, the growing complexity of artificial intelligence (AI) algorithms has led to a massive increase in the computing power required for training these systems. From 2010 to 2020, the compute demand for AI training increased by a factor of 10,000.
Neuromorphic computing, which uses a more brain-like architecture with integrated memory and processing, could help address these issues. But the key hardware element to enable this has not yet been developed. This paper explores the potential of non-linear resistive switches as the building block for neuromorphic computers.
Technical Explanation
The paper discusses the exponential growth in data processing by information and communications technologies (ICT), driven by applications like cryptocurrency mining and search engines. This has led to a significant increase in the energy consumption of ICT systems, which is expected to rise from 8% of global electricity use in 2010 to 14% by 2030.
A major contributor to this high energy demand is the von-Neumann architecture used in traditional computing devices, which separates memory and processor units. This separation results in constant data transfer between the two, which not only consumes a lot of energy but also severely limits the computing power of these systems.
Additionally, the paper notes that the increasing complexity of AI algorithms has led to a dramatic increase in the computing power required for training these systems. From 2010 to 2020, the compute demand for AI training increased by a factor of 10,000.
To address these issues, the paper discusses the potential of neuromorphic computing, which uses a more brain-like architecture with integrated memory and processing units. However, the core hardware element required for this approach has not yet been realized.
The paper proposes that non-linear resistive switches could be the key to enabling neuromorphic computing, as they could serve as the building blocks for merged memory and processor units.
Critical Analysis
The paper provides a compelling overview of the significant energy challenges facing the ICT industry due to the exponential growth in data processing and the limitations of the von-Neumann architecture. The authors make a strong case for the potential of neuromorphic computing to address these issues.
However, the paper acknowledges that the core hardware element required for neuromorphic computing has not yet been developed. The focus on non-linear resistive switches as a potential solution is an interesting avenue for further research, but the paper does not provide any details on the specific design or performance characteristics of these devices.
It would be helpful to see more experimental data or simulations demonstrating the feasibility and advantages of non-linear resistive switches for neuromorphic computing. Additionally, the paper could explore potential challenges or limitations in implementing this approach, such as scalability, reliability, or integration with existing semiconductor manufacturing processes.
Further research and development in this area could have significant implications for the energy efficiency and computing performance of future ICT systems, but the practical realization of neuromorphic computing based on the proposed hardware elements remains to be demonstrated.
Conclusion
This paper highlights the growing energy challenges faced by the information and communications technology (ICT) industry due to the exponential increase in data processing, driven by applications such as cryptocurrency mining and search engines. The energy consumption of ICT systems is expected to rise from 8% of global electricity use in 2010 to 14% by 2030, largely due to the limitations of the von-Neumann architecture, which separates memory and processing units.
The paper proposes that neuromorphic computing, which uses a more brain-like architecture with integrated memory and processing, could help address these energy and performance challenges. However, the core hardware element required for this approach has not yet been realized.
The authors suggest that non-linear resistive switches could be the key to enabling neuromorphic computing, as they could serve as the building blocks for merged memory and processor units. Further research and development in this area could have significant implications for the future of energy-efficient and high-performance computing in the ICT sector.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
💬
Spintronic memristors for computing
Qiming Shao, Zhongrui Wang, Yan Zhou, Shunsuke Fukami, Damien Querlioz, Yiran Chen, Leon O. Chua
0
0
The ever-increasing amount of data from ubiquitous smart devices fosters data-centric and cognitive algorithms. Traditional digital computer systems have separate logic and memory units, resulting in a huge delay and energy cost for implementing these algorithms. Memristors are programmable resistors with a memory, providing a paradigm-shifting approach towards creating intelligent hardware systems to handle data-centric tasks. Spintronic nanodevices are promising choices as they are high-speed, low-power, highly scalable, robust, and capable of constructing dynamic complex systems. In this Review, we survey spintronic devices from a memristor point of view. We introduce spintronic memristors based on magnetic tunnel junctions, nanomagnet ensemble, domain walls, topological spin textures, and spin waves, which represent dramatically different state spaces. They can exhibit steady, oscillatory, stochastic, and chaotic trajectories in their state spaces, which have been exploited for in-memory logic, neuromorphic computing, stochastic and chaos computing. Finally, we discuss challenges and trends in realizing large-scale spintronic memristive systems for practical applications.
4/23/2024
🎯
Machine Learning Without a Processor: Emergent Learning in a Nonlinear Electronic Metamaterial
Sam Dillavou, Benjamin D Beyer, Menachem Stern, Andrea J Liu, Marc Z Miskin, Douglas J Durian
0
0
Standard deep learning algorithms require differentiating large nonlinear networks, a process that is slow and power-hungry. Electronic learning metamaterials offer potentially fast, efficient, and fault-tolerant hardware for analog machine learning, but existing implementations are linear, severely limiting their capabilities. These systems differ significantly from artificial neural networks as well as the brain, so the feasibility and utility of incorporating nonlinear elements have not been explored. Here we introduce a nonlinear learning metamaterial -- an analog electronic network made of self-adjusting nonlinear resistive elements based on transistors. We demonstrate that the system learns tasks unachievable in linear systems, including XOR and nonlinear regression, without a computer. We find our nonlinear learning metamaterial reduces modes of training error in order (mean, slope, curvature), similar to spectral bias in artificial neural networks. The circuitry is robust to damage, retrainable in seconds, and performs learned tasks in microseconds while dissipating only picojoules of energy across each transistor. This suggests enormous potential for fast, low-power computing in edge systems like sensors, robotic controllers, and medical devices, as well as manufacturability at scale for performing and studying emergent learning.
4/9/2024
🐍
Towards Reverse-Engineering the Brain: Brain-Derived Neuromorphic Computing Approach with Photonic, Electronic, and Ionic Dynamicity in 3D integrated circuits
S. J. Ben Yoo, Luis El-Srouji, Suman Datta, Shimeng Yu, Jean Anne Incorvia, Alberto Salleo, Volker Sorger, Juejun Hu, Lionel C Kimerling, Kristofer Bouchard, Joy Geng, Rishidev Chaudhuri, Charan Ranganath, Randall O'Reilly
0
0
The human brain has immense learning capabilities at extreme energy efficiencies and scale that no artificial system has been able to match. For decades, reverse engineering the brain has been one of the top priorities of science and technology research. Despite numerous efforts, conventional electronics-based methods have failed to match the scalability, energy efficiency, and self-supervised learning capabilities of the human brain. On the other hand, very recent progress in the development of new generations of photonic and electronic memristive materials, device technologies, and 3D electronic-photonic integrated circuits (3D EPIC ) promise to realize new brain-derived neuromorphic systems with comparable connectivity, density, energy-efficiency, and scalability. When combined with bio-realistic learning algorithms and architectures, it may be possible to realize an 'artificial brain' prototype with general self-learning capabilities. This paper argues the possibility of reverse-engineering the brain through architecting a prototype of a brain-derived neuromorphic computing system consisting of artificial electronic, ionic, photonic materials, devices, and circuits with dynamicity resembling the bio-plausible molecular, neuro/synaptic, neuro-circuit, and multi-structural hierarchical macro-circuits of the brain based on well-tested computational models. We further argue the importance of bio-plausible local learning algorithms applicable to the neuromorphic computing system that capture the flexible and adaptive unsupervised and self-supervised learning mechanisms central to human intelligence. Most importantly, we emphasize that the unique capabilities in brain-derived neuromorphic computing prototype systems will enable us to understand links between specific neuronal and network-level properties with system-level functioning and behavior.
4/1/2024
Measurement-driven neural-network training for integrated magnetic tunnel junction arrays
William A. Borders, Advait Madhavan, Matthew W. Daniels, Vasileia Georgiou, Martin Lueker-Boden, Tiffany S. Santos, Patrick M. Braganca, Mark D. Stiles, Jabez J. McClelland, Brian D. Hoskins
0
0
The increasing scale of neural networks needed to support more complex applications has led to an increasing requirement for area- and energy-efficient hardware. One route to meeting the budget for these applications is to circumvent the von Neumann bottleneck by performing computation in or near memory. An inevitability of transferring neural networks onto hardware is that non-idealities such as device-to-device variations or poor device yield impact performance. Methods such as hardware-aware training, where substrate non-idealities are incorporated during network training, are one way to recover performance at the cost of solution generality. In this work, we demonstrate inference on hardware neural networks consisting of 20,000 magnetic tunnel junction arrays integrated on a complementary metal-oxide-semiconductor chips that closely resembles market-ready spin transfer-torque magnetoresistive random access memory technology. Using 36 dies, each containing a crossbar array with its own non-idealities, we show that even a small number of defects in physically mapped networks significantly degrades the performance of networks trained without defects and show that, at the cost of generality, hardware-aware training accounting for specific defects on each die can recover to comparable performance with ideal networks. We then demonstrate a robust training method that extends hardware-aware training to statistics-aware training, producing network weights that perform well on most defective dies regardless of their specific defect locations. When evaluated on the 36 physical dies, statistics-aware trained solutions can achieve a mean misclassification error on the MNIST dataset that differs from the software-baseline by only 2 %. This statistics-aware training method could be generalized to networks with many layers that are mapped to hardware suited for industry-ready applications.
5/15/2024