NaviSplit: Dynamic Multi-Branch Split DNNs for Efficient Distributed Autonomous Navigation
0
Sign in to get full access
Overview
- Dynamic multi-branch split deep neural networks (DNNs) for efficient distributed autonomous navigation
- Supervised compression technique to split a single large DNN into multiple smaller sub-networks
- Enables adaptive resource allocation and distributed inference on resource-constrained edge devices
- Improves energy efficiency and inference latency compared to a single monolithic DNN
Plain English Explanation
The paper presents a system called NaviSplit that aims to make autonomous navigation more efficient on resource-constrained devices. Autonomous navigation, such as self-driving cars or robots, often relies on large and complex deep neural networks (DNNs) to process sensor data and make decisions. However, running these large DNNs on edge devices with limited computing power can be challenging.
NaviSplit solves this problem by taking a single large DNN and splitting it into multiple smaller sub-networks using a supervised compression technique. These smaller sub-networks can then be deployed across multiple edge devices, allowing for distributed inference and more efficient use of computing resources. This enables the system to adapt to the specific hardware capabilities of each device, improving energy efficiency and reducing inference latency compared to running a single monolithic DNN.
The key innovation is the ability to dynamically adjust the allocation of these sub-networks across the distributed system based on the current workload and resource availability. This allows the autonomous navigation system to maintain high performance even as the compute resources change, such as when some devices are turned off or become overloaded.
Technical Explanation
The NaviSplit system uses a supervised compression technique to split a single large DNN into multiple smaller sub-networks, or "branches." This is done by training the sub-networks to collectively mimic the behavior of the original DNN, ensuring they retain the necessary functionality for autonomous navigation tasks.
The architecture of NaviSplit consists of a Splitting Module that divides the original DNN, and a Coordination Module that dynamically allocates the sub-networks across the distributed system. The Coordination Module continuously monitors the resource usage and performance of each sub-network, and adjusts the allocation accordingly to optimize for factors like energy efficiency and inference latency.
The researchers evaluated NaviSplit on several autonomous navigation benchmarks, including SayCan: Grounding Language to Large-Scale Commonsense Knowledge for Robots, Collaborative Satellite Computing through Adaptive DNN Task, and Resource-Aware Deployment of Dynamic DNNs over Multi. The results showed that NaviSplit can achieve significant improvements in energy efficiency and inference latency compared to a single monolithic DNN, while maintaining comparable navigation performance.
Critical Analysis
The paper provides a comprehensive evaluation of NaviSplit and addresses several important practical considerations for deploying autonomous navigation systems on resource-constrained edge devices. However, there are a few potential limitations and areas for further research:
-
The paper focuses on supervised compression, which may not be suitable for all types of DNNs or tasks. Exploring unsupervised or reinforcement learning-based methods for splitting DNNs could further expand the applicability of the approach.
-
The dynamic allocation of sub-networks assumes that the resource usage and performance of each sub-network can be accurately measured and predicted. In real-world scenarios, there may be additional factors, such as network latency or sensor data quality, that could affect the system's behavior.
-
The paper does not discuss the potential for sub-networks to diverge or become inconsistent over time, which could lead to coordination issues or degraded navigation performance. Mechanisms to ensure the sub-networks remain aligned would be an important area of investigation.
-
While the paper showcases the benefits of NaviSplit on specific autonomous navigation benchmarks, it would be valuable to see how the system performs on a wider range of tasks and environments, including Online Robot Navigation and Manipulation with Distilled Vision-Language and Autonomous Driving with Deep Dual Model Solution for Steering.
Conclusion
The NaviSplit system presents a promising approach for improving the efficiency of autonomous navigation on resource-constrained edge devices. By dynamically splitting a single large DNN into multiple smaller sub-networks and adaptively allocating them across a distributed system, NaviSplit can achieve significant gains in energy efficiency and inference latency while maintaining comparable navigation performance. The supervised compression technique and dynamic coordination module are key innovations that address practical challenges in deploying complex AI systems on the edge. While the paper identifies several areas for further research, NaviSplit demonstrates the potential for dynamic and distributed DNN architectures to enable more efficient and capable autonomous navigation systems.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
NaviSplit: Dynamic Multi-Branch Split DNNs for Efficient Distributed Autonomous Navigation
Timothy K Johnsen, Ian Harshbarger, Zixia Xia, Marco Levorato
Lightweight autonomous unmanned aerial vehicles (UAV) are emerging as a central component of a broad range of applications. However, autonomous navigation necessitates the implementation of perception algorithms, often deep neural networks (DNN), that process the input of sensor observations, such as that from cameras and LiDARs, for control logic. The complexity of such algorithms clashes with the severe constraints of these devices in terms of computing power, energy, memory, and execution time. In this paper, we propose NaviSplit, the first instance of a lightweight navigation framework embedding a distributed and dynamic multi-branched neural model. At its core is a DNN split at a compression point, resulting in two model parts: (1) the head model, that is executed at the vehicle, which partially processes and compacts perception from sensors; and (2) the tail model, that is executed at an interconnected compute-capable device, which processes the remainder of the compacted perception and infers navigation commands. Different from prior work, the NaviSplit framework includes a neural gate that dynamically selects a specific head model to minimize channel usage while efficiently supporting the navigation network. In our implementation, the perception model extracts a 2D depth map from a monocular RGB image captured by the drone using the robust simulator Microsoft AirSim. Our results demonstrate that the NaviSplit depth model achieves an extraction accuracy of 72-81% while transmitting an extremely small amount of data (1.2-18 KB) to the edge server. When using the neural gate, as utilized by NaviSplit, we obtain a slightly higher navigation accuracy as compared to a larger static network by 0.3% while significantly reducing the data rate by 95%. To the best of our knowledge, this is the first exemplar of dynamic multi-branched model based on split DNNs for autonomous navigation.
Read more6/21/2024
0
Distilling Tiny and Ultra-fast Deep Neural Networks for Autonomous Navigation on Nano-UAVs
Lorenzo Lamberti, Lorenzo Bellone, Luka Macan, Enrico Natalizio, Francesco Conti, Daniele Palossi, Luca Benini
Nano-sized unmanned aerial vehicles (UAVs) are ideal candidates for flying Internet-of-Things smart sensors to collect information in narrow spaces. This requires ultra-fast navigation under very tight memory/computation constraints. The PULP-Dronet convolutional neural network (CNN) enables autonomous navigation running aboard a nano-UAV at 19 frame/s, at the cost of a large memory footprint of 320 kB -- and with drone control in complex scenarios hindered by the disjoint training of collision avoidance and steering capabilities. In this work, we distill a novel family of CNNs with better capabilities than PULP-Dronet, but memory footprint reduced by up to 168x (down to 2.9 kB), achieving an inference rate of up to 139 frame/s; we collect a new open-source unified collision/steering 66 k images dataset for more robust navigation; and we perform a thorough in-field analysis of both PULP-Dronet and our tiny CNNs running on a commercially available nano-UAV. Our tiniest CNN, called Tiny-PULP-Dronet v3, navigates with a 100% success rate a challenging and never-seen-before path, composed of a narrow obstacle-populated corridor and a 180{deg} turn, at a maximum target speed of 0.5 m/s. In the same scenario, the SoA PULP-Dronet consistently fails despite having 168x more parameters.
Read more7/18/2024
🌀
0
MTL-Split: Multi-Task Learning for Edge Devices using Split Computing
Luigi Capogrosso, Enrico Fraccaroli, Samarjit Chakraborty, Franco Fummi, Marco Cristani
Split Computing (SC), where a Deep Neural Network (DNN) is intelligently split with a part of it deployed on an edge device and the rest on a remote server is emerging as a promising approach. It allows the power of DNNs to be leveraged for latency-sensitive applications that do not allow the entire DNN to be deployed remotely, while not having sufficient computation bandwidth available locally. In many such embedded systems scenarios, such as those in the automotive domain, computational resource constraints also necessitate Multi-Task Learning (MTL), where the same DNN is used for multiple inference tasks instead of having dedicated DNNs for each task, which would need more computing bandwidth. However, how to partition such a multi-tasking DNN to be deployed within a SC framework has not been sufficiently studied. This paper studies this problem, and MTL-Split, our novel proposed architecture, shows encouraging results on both synthetic and real-world data. The source code is available at https://github.com/intelligolabs/MTL-Split.
Read more7/9/2024
0
Resource-aware Deployment of Dynamic DNNs over Multi-tiered Interconnected Systems
Chetna Singhal, Yashuo Wu, Francesco Malandrino, Marco Levorato, Carla Fabiana Chiasserini
The increasing pervasiveness of intelligent mobile applications requires to exploit the full range of resources offered by the mobile-edge-cloud network for the execution of inference tasks. However, due to the heterogeneity of such multi-tiered networks, it is essential to make the applications' demand amenable to the available resources while minimizing energy consumption. Modern dynamic deep neural networks (DNN) achieve this goal by designing multi-branched architectures where early exits enable sample-based adaptation of the model depth. In this paper, we tackle the problem of allocating sections of DNNs with early exits to the nodes of the mobile-edge-cloud system. By envisioning a 3-stage graph-modeling approach, we represent the possible options for splitting the DNN and deploying the DNN blocks on the multi-tiered network, embedding both the system constraints and the application requirements in a convenient and efficient way. Our framework -- named Feasible Inference Graph (FIN) -- can identify the solution that minimizes the overall inference energy consumption while enabling distributed inference over the multi-tiered network with the target quality and latency. Our results, obtained for DNNs with different levels of complexity, show that FIN matches the optimum and yields over 65% energy savings relative to a state-of-the-art technique for cost minimization.
Read more4/15/2024