3DTINC: Time-Equivariant Non-Contrastive Learning for Predicting Disease Progression from Longitudinal OCTs

Overview

• This paper proposes a new self-supervised learning (SSL) method called 3DTINC for learning representations from 3D optical coherence tomography (OCT) volumes of the retina.
• Existing contrastive SSL methods require large batch sizes and augmentations designed for natural images, which are impractical for 3D medical images like OCT.
• 3DTINC uses a non-contrastive similarity loss to learn perturbation-invariant features from intra-patient OCT scans acquired at different timepoints, capturing temporal information crucial for predicting disease progression.
• The authors evaluate the learned representations on two large-scale longitudinal datasets for predicting conversion to wet age-related macular degeneration (AMD), demonstrating the importance of their contributions.

Plain English Explanation

Self-supervised learning is a powerful technique for improving the performance of deep learning models, especially when labeled data is scarce. Contrastive methods are a prominent family of SSL that learn similar representations for two modified versions of an image, while pushing away representations of other images as negatives.

However, existing contrastive SSL methods have limitations when applied to 3D medical images like optical coherence tomography (OCT) of the retina. They require large batches of images and data augmentations designed for natural images, which are not suitable for the 3D structure and noise characteristics of OCT scans.

To address these challenges, the researchers propose a new SSL method called 3DTINC that uses a non-contrastive approach. Instead of pushing away negative examples, 3DTINC learns perturbation-invariant features by exploiting the temporal information inherent in longitudinal OCT scans of the same patient acquired at different timepoints.

The key insight is that the changes in a patient's retinal structure over time, even if subtle, contain valuable information about the progression of diseases like age-related macular degeneration (AMD). By learning to extract features that are consistent across these intra-patient scans, 3DTINC can capture this temporal information, which is crucial for predicting disease progression.

Technical Explanation

The 3DTINC method is designed to learn representations from 3D OCT volumes that are robust to perturbations and can capture temporal information relevant for predicting disease progression. It differs from contrastive SSL approaches in several ways:

1. Non-contrastive learning: Instead of pushing away "negative" examples in the representation space, 3DTINC uses a novel non-contrastive similarity loss that encourages the model to learn features that are consistent across different timepoints for the same patient, as opposed to across different patients.

2. Temporal information: By leveraging the longitudinal nature of OCT scans, 3DTINC can implicitly learn temporal information about disease progression, which is crucial for predicting outcomes like conversion to wet AMD.

3. Targeted augmentations: The authors design data augmentations specifically for 3D OCT volumes, such as applying random rotations, scaling, and noise, to ensure the learned representations are robust to the unique characteristics of this modality.

The authors evaluate the representations learned by 3DTINC on two large-scale longitudinal datasets of retinal OCT scans, where the task is to predict the conversion to wet AMD within a 6-month interval. They show that each component of their approach - the non-contrastive loss, the use of temporal information, and the specialized augmentations - is crucial for achieving state-of-the-art performance on this prognostic task.

Critical Analysis

The 3DTINC method addresses important limitations of existing contrastive SSL approaches when applied to 3D medical imaging data. By focusing on learning perturbation-invariant features from longitudinal scans, the authors demonstrate the value of incorporating temporal information, which is often overlooked in computer vision tasks.

However, a potential limitation of the work is the reliance on intra-patient scans, which may not always be available, especially for rare diseases. An interesting direction for future research could be to explore ways of leveraging inter-patient information, perhaps through anatomical conditioning or adapting self-supervised learning to computational pathology.

Additionally, the authors could further investigate the interpretability of the learned representations and how they capture clinically relevant features for disease progression. Techniques like LATIM could be leveraged to better understand the temporal dynamics encoded in the representations.

Overall, the 3DTINC method is a promising approach for learning robust representations from 3D medical imaging data, with potential applications in a wide range of prognostic tasks beyond the specific case of AMD studied in this work.

Conclusion

The 3DTINC method proposed in this paper addresses key limitations of existing contrastive self-supervised learning techniques when applied to 3D medical imaging data, such as optical coherence tomography (OCT) of the retina. By leveraging the temporal information inherent in longitudinal scans and using specialized data augmentations, 3DTINC can learn perturbation-invariant features that are crucial for predicting disease progression, as demonstrated by its state-of-the-art performance on predicting conversion to wet age-related macular degeneration.

This work highlights the importance of tailoring self-supervised learning approaches to the unique characteristics of medical imaging modalities, and the potential of exploiting temporal information for prognostic tasks. The insights from this research could inspire further advancements in self-supervised representation learning for 3D medical imaging, with broader implications for improving the efficiency and effectiveness of deep learning in healthcare applications.