Colorful Cutout: Enhancing Image Data Augmentation with Curriculum Learning

Introduction

This paper discusses a new data augmentation technique for deep learning models that aims to improve their generalization ability and prevent overfitting. The authors note that traditional data augmentation techniques, such as cropping, rotating, and jittering, have evolved to include more complex methods like cutout and random erasing. These techniques randomly erase portions of the input image to act as a dropout strategy. The authors also mention mixup and cutmix, which combine multiple images through mixing and cutting.

However, the authors argue that previous approaches have not considered the difficulty of the augmented data. They suggest that a well-defined training procedure that takes into account the difficulty of the data can enhance model performance. To this end, the authors propose a novel curriculum data augmentation technique for image data.

Their method introduces colorization into the cutout process, allowing them to control the difficulty of the augmented images by dividing the erasure box and filling the sub-regions with different colors. The authors claim this is the first study to pioneer curriculum data augmentation in computer vision.

The paper presents comprehensive experiments on various models and datasets, demonstrating the effectiveness of the proposed curriculum data augmentation technique.

Method

The text describes a new data augmentation technique called "colorful cutout". This approach builds on the traditional cutout method, which randomly selects a box region from an image and replaces it with zeros. Instead of simple erasure, the colorful cutout fills the box with a random color. This adds more variation to the augmented images compared to previous methods, leading to performance gains.

The colorful cutout also introduces a curriculum learning approach. The erasure box is divided into sub-regions, each with a different random color. As training progresses, the number of sub-regions increases, making the erasure box more complex and the resulting samples more difficult. This gradual increase in difficulty is demonstrated in Figure 1. The appendix provides the pseudocode for the colorful cutout technique.

Experiment

The experiment evaluated the effectiveness of the proposed colorful cutout method. Three different datasets were used for the evaluation: CIFAR-10, CIFAR-100, and Tiny ImageNet. The proposed method was compared against various previous augmentation techniques, including traditional cutout, mixup, and cutmix. The experiments were conducted on three different models: CNN-based ResNet50 and EfficientNet-B0, and Transformer-based ViT-B/16.

The results demonstrate a significant improvement in model performance with colorful cutout compared to other methods, particularly traditional cutout. An ablation experiment on colorful cutout without the curriculum data augmentation showed similar performance to cutout, suggesting the curriculum data augmentation plays an important role in enhancing the model's performance. This indicates the potential of curriculum data augmentation in image data augmentation.

Conclusion

The paper proposes a simple yet effective data augmentation strategy that incorporates the concept of curriculum learning into computer vision tasks. Experimental results demonstrate the effectiveness of this approach and the potential for curriculum-based image augmentation. The paper suggests future research could investigate applying curriculum data augmentation to other image augmentation strategies and introducing soft labels to augmented data based on difficulty.

The research was supported by funding from the National Research Foundation of Korea and the Institute for Information & communications Technology Planning & Evaluation. The first author Juhwan Choi meets the underrepresented minority criteria for the ICLR 2024 Tiny Papers Track.

Appendix A Implementation Details

This section provides details on the implementation and setup used for reproducing the results in the paper. The key points are:

Model Implementation:

• Three models were used, based on pre-trained ImageNet checkpoints from the TorchVision library.
• After feature extraction, a two-layer classification head with dropout and ReLU activation was added.
• Input images were resized to 256x256 and randomly cropped to 224x224 during training. In validation/test, a 224x224 center crop was used.

Augmentation Implementation:

• For traditional cutout, cutmix, and the proposed colorful cutout, the cutout box size was set to 32x32.
• For mixup and cutmix, the alpha hyperparameter was set to 0.2.
• Colorful cutout increases the number of sub-regions over training epochs, starting with 0 sub-regions in the first epoch.

Datasets:

• CIFAR-10, CIFAR-100, and Tiny ImageNet were used, downloaded from the Hugging Face Datasets library.
• 10% of the training data was randomly selected for the validation set, as no predefined validation set exists.

Hyperparameters:

• Adam optimizer was used with a learning rate of 5e-5.
• Models were trained for 5 epochs with a batch size of 32.
• Label smoothing with a factor of 0.05 was applied.

Other Details:

• Experiments were run on a single NVIDIA RTX 3090 GPU.
• Training times were 75.7 minutes for colorful cutout on Tiny ImageNet, and 74.2 minutes for the cutout baseline.

Appendix B Comparison between Other Techniques

The provided text does not contain enough information to summarize. No text was provided to summarize.

Appendix C Algorithm of Colorful Cutout

The text provided gives a pseudo-code for implementing a "colorful cutout" technique. This approach allows creating images with cutouts of various shapes and colors. The pseudo-code outlines the key steps involved, including loading an input image, defining the cutout shapes, assigning colors to the cutouts, and compositing the final image. The technique provides a flexible way to generate visually striking images by combining different cutout shapes and colors.