SWANN: Shuffling Weights in Crossbar Arrays for Enhanced DNN Accuracy in Deeply Scaled Technologies

Overview

• Researchers propose a technique called SWANN to address the problem of interconnect resistance impairing the robustness of in-memory computing (IMC) in deep neural network (DNN) accelerators using crossbar arrays.
• SWANN involves shuffling weights in crossbar arrays to alleviate the detrimental effect of wire resistance on IMC.
• The technique is evaluated on 8T-SRAM-based 128x128 crossbar arrays in 7nm technology, showing an increase in accuracy for ResNet-20/CIFAR-10 from 47.78% to 83.5%.
• SWANN can be used in conjunction with Partial-Word-LineActivation to further boost accuracy.
• The impact of SWANN is also evaluated for compact ferroelectric-transistor-based crossbar arrays.
• SWANN incurs minimal hardware overhead, with less than a 1% increase in energy consumption and around 1% latency and 16% area overheads.

Plain English Explanation

Deep neural networks (DNNs) are powerful machine learning models that can excel at tasks like image recognition and natural language processing. To run these models efficiently, researchers have developed specialized hardware called DNN accelerators. These accelerators often use crossbar arrays, which are grid-like structures that can perform in-memory computing – a way of doing calculations directly within the memory, rather than moving data back and forth.

However, as these crossbar arrays are made smaller and more compact using advanced manufacturing techniques, the wires connecting the different components can start to interfere with each other, leading to a drop in the accuracy of the DNN. This is a significant problem that needs to be solved.

The researchers propose a technique called SWANN (Shuffling Weights in Analog Neural Networks) to address this issue. The key idea behind SWANN is to rearrange, or "shuffle," the weights (the parameters that determine how the DNN model behaves) within the crossbar array in a specific way. This shuffling helps to counteract the negative effects of the wire resistance, ultimately improving the accuracy of the DNN.

The researchers show that SWANN can significantly boost the accuracy of a ResNet-20 DNN model running on CIFAR-10 data, from 47.78% to 83.5%. They also demonstrate that SWANN can be used together with another technique called Partial-Word-LineActivation to further improve the performance.

Importantly, SWANN achieves these benefits with minimal additional hardware overhead, requiring less than a 1% increase in energy consumption and around 1% in latency and 16% in area. This makes it a practical and efficient solution for improving the performance of DNN accelerators.

Technical Explanation

The researchers focus on the problem of interconnect resistance severely impairing the robustness of in-memory computing (IMC) in deep neural network (DNN) accelerators that employ crossbar arrays. In deeply scaled technologies, the high resistance of the interconnects between the components in the crossbar array can significantly degrade the accuracy of the DNN computations performed within the array.

To address this issue, the researchers propose a technique called SWANN (Shuffling Weights in Analog Neural Networks). SWANN involves rearranging the weights (the parameters that determine the behavior of the DNN model) within the crossbar array in a specific way. This weight shuffling helps to counteract the negative effects of the wire resistance, ultimately improving the accuracy of the DNN.

The researchers evaluate SWANN on 8T-SRAM-based 128x128 crossbar arrays in 7nm technology. For a ResNet-20 DNN model running on the CIFAR-10 dataset, SWANN enhances the accuracy from 47.78% to 83.5%. The researchers also demonstrate that SWANN can be used synergistically with another technique called Partial-Word-LineActivation, further boosting the accuracy.

Additionally, the researchers evaluate the implications of SWANN for compact ferroelectric-transistor-based crossbar arrays. They find that SWANN incurs minimal hardware overhead, with less than a 1% increase in energy consumption and around 1% latency and 16% area overheads when 1 ADC (Analog-to-Digital Converter) is utilized per crossbar array.

Critical Analysis

The researchers have presented a promising technique, SWANN, to address the significant problem of interconnect resistance degrading the performance of in-memory computing in DNN accelerators. The key strength of SWANN is its ability to effectively mitigate the detrimental effects of wire resistance without incurring substantial hardware overhead.

However, the paper does not provide a detailed analysis of the limitations or potential downsides of the SWANN approach. For instance, it would be valuable to understand the impact of SWANN on other important metrics, such as power consumption or chip area, beyond the specific cases presented.

Additionally, the researchers focus primarily on evaluating SWANN on 8T-SRAM-based and ferroelectric-transistor-based crossbar arrays. It would be interesting to see how SWANN performs in the context of other crossbar array architectures, such as those using symmetric silicon microring resonator optical crossbar arrays or tiny shared block efficient DNN deployment techniques.

Furthermore, the paper does not discuss the potential implications of SWANN on the training process or the overall system design of DNN accelerators. It would be valuable to understand how SWANN might interact with other techniques, such as quantized spiking neural networks or measurement-driven neural network training, and how it could be integrated into a broader system-level optimization.

Despite these potential areas for further research, the SWANN technique represents a significant contribution to the field of DNN accelerators, demonstrating the ability to effectively mitigate the challenges posed by interconnect resistance in deeply scaled technologies.

Conclusion

The researchers have proposed a novel technique called SWANN (Shuffling Weights in Analog Neural Networks) to address the problem of interconnect resistance impairing the robustness of in-memory computing in deep neural network (DNN) accelerators using crossbar arrays. SWANN involves rearranging the weights within the crossbar array to counteract the negative effects of wire resistance, leading to a substantial improvement in DNN accuracy.

The authors have demonstrated the effectiveness of SWANN on 8T-SRAM-based and ferroelectric-transistor-based crossbar arrays, showing significant accuracy gains with minimal hardware overhead. The ability to integrate SWANN with other techniques, such as Partial-Word-LineActivation, further enhances its potential for improving the performance of DNN accelerators.

The SWANN approach represents an important advancement in the field of DNN hardware design, addressing a crucial challenge posed by the scaling of interconnect technologies. As the demand for efficient and high-performing DNN accelerators continues to grow, techniques like SWANN will be essential for enabling the deployment of these powerful models in real-world applications.