ENet-21: An Optimized light CNN Structure for Lane Detection

Introduction

The paper discusses the application of computer vision techniques for lane detection in autonomous vehicles. Two primary methods for lane detection are discussed: classical algorithms and machine learning-based algorithms. Classical algorithms rely on handmade feature selection and are dependent on the quality of input images. Machine learning-based algorithms, particularly Convolutional Neural Networks (CNNs), can perform automatic feature extraction and provide a more robust solution against fluctuating elements and variable environments.

Recent studies have implemented end-to-end approaches using CNNs without any initial preprocessing. The Spatial Convolutional Neural Network (SCNN) is introduced to address performance degradation caused by obstacles blocking lane markings. Multi-task CNNs are proposed to detect multiple lanes in one image by performing both segmentation and embedding branches.

The main challenge with deep learning-based approaches is the computational cost associated with the complexity of CNN structures. This can be addressed by running the algorithm on suitable hardware or reducing the network's complexity.

The contributions of the study include:

1. Presenting a light CNN backbone with fewer FLOPs and parameters while achieving similar performance to existing architectures.
2. Using affinity fields for segmentation to effectively group and link pixels belonging to shapeless entities like lane markings.
3. Demonstrating performance comparable to or surpassing leading techniques in the TuSimple benchmark dataset without complicated post-processing.

The study employs a machine learning-based method using CNN for feature extraction on the TuSimple dataset and proposes a light CNN structure for lane detection problems.

Related Works

The provided section reviews recent advancements in lane detection using deep learning methods. Lane detection aims to accurately identify and differentiate lane shapes. Current deep learning-based methods can be categorized based on their lane modeling approach, including semantic and instance segmentation, key point detection, gridding, polynomial, and anchor-based methods.

The paper discusses several specific methods:

• Focus on Local uses key point detection and local geometry construction to rectify key point pixel locations.
• PINet utilizes the Stacked Hourglass Network for key point detection and predicts offsets in x and y directions.
• CondLaneNet's CNN is based on conditional instance segmentation and row anchor formulation but struggles with recognizing start points in complex scenarios.
• SCNN treats lane detection as multiclass semantic segmentation and uses a spatial CNN module to aggregate spatial information, but it is not real-time due to low inference speed.
• LaneNet employs an instance segmentation pipeline to handle a variable number of lanes but requires post-inference clustering.
• Line-CNN is an end-to-end lane detector based on Faster R-CNN but struggles with real-time performance due to high latency.
• LaneATT introduces a novel anchor-based attention technique that aggregates global information, achieving good performance, adequacy, and efficiency.

The section highlights the challenges and limitations of various methods, such as handling complex scenarios, real-time performance, and the need for post-processing steps.

Methodology

The paper presents a new methodology for lane marker detection using an end-to-end convolutional neural network (CNN). The approach treats lane prediction as a segmentation problem, training the model to predict semantic segmentation labels and pixel-wise affinity fields. These affinity fields are divided into horizontal (HAF) and vertical (VAF) components, which map 2D locations of the input image into 2D unit vectors. The VAF encodes lane pixel vector direction toward the next lane pixels in the above row, while the HAF points toward the center pixel of the lane in the current row. In a postprocessing step, the VAF, HAF, and semantic lane segmentation enable lane generation through foreground pixel clustering. The main focus is on developing a lighter CNN architecture, called the backbone, that maintains proper performance. The proposed architecture for binary segmentation and affinity fields, along with its training and inference procedure, will be discussed in the following subsections.

The paper proposes a lane detection model using an efficient neural network architecture called ENet as the backbone. The model predicts both a binary segmentation mask and affinity fields to cluster lane pixels horizontally and vertically.

The ENet architecture is divided into encoder and decoder stages, with bottleneck blocks containing projection, main, and expansion convolutional layers. Batch normalization and PReLU activation are used after each convolutional layer. The model has fewer parameters than other popular architectures like ResNet and ERFNet.

The affinity fields, consisting of a horizontal affinity field (HAF) and vertical affinity field (VAF), are generated using ground truth masks. The HAF assigns unit vectors pointing to the center of each lane in a row, while the VAF assigns unit vectors pointing to the center of each lane in the previous row. During testing, a decoding process clusters foreground pixels into lanes using the predicted affinity fields.

The model is trained using a loss function that combines weighted binary cross-entropy loss for semantic segmentation, intersection over union loss to handle the imbalanced dataset, and L1 regression loss for the affinity field heads. The general loss function is the sum of these individual losses.

Experimental Results

The paper evaluates a lane detection approach using the TuSimple dataset, which contains 6408 annotated highway images. The evaluation metrics used are accuracy (percentage of correctly predicted lane vertices), false positive rate, false negative rate, and F1-score.

The proposed method resizes input images to 640x352 and generates lane marker masks based on JSON labels. It uses an Adam optimizer with a learning rate of 0.0005, mini-batches of 8 images per GPU, and applies techniques like dropout, early stopping, and data augmentation to reduce overfitting.

Compared to state-of-the-art methods, the proposed architecture achieves competitive results on the TuSimple dataset, with a notable F1-score and low false positive rate. The model is lightweight, with only 0.25 million parameters and 3.14G FLOPs, making it faster than commonly used models while maintaining similar accuracy.

Ablation experiments were performed to validate configuration choices, investigating the impact of hyper-parameters, layer configurations, and noise types. The optimized architecture balances performance and efficiency for lane detection tasks.

Conclusion

The study introduces a new CNN architecture for end-to-end lane detection that uses semantic segmentation and affinity fields. The method employs binary segmentation outputs and affinity fields to perform instance segmentation of lane pixels, enabling the detection of varying numbers of lanes in post-processing. The proposed architecture has fewer parameters than most other models and outperforms them, achieving a low false positive rate on the TuSimple dataset. The model also has the lowest computational complexity, including FLOPs and number of parameters, among existing models. The work offers a promising solution for lane detection in autonomous driving applications.

The authors declare that they received no financial support during the preparation of the manuscript.

ompliance with ethical standards

The authors, Seyed Rasoul Hosseini and Mohammad Teshnehlab, state they have no conflicts of interest related to the study. No experiments involving humans or animals were conducted. The study obtained informed consent from all participants.