A self-supervised framework for learning whole slide representations
0
Sign in to get full access
Related Work
Computational Pathology
Computational pathology is an emerging field that applies machine learning and artificial intelligence to the analysis of histopathological images, such as those obtained from whole slide imaging (WSI) of tissue samples. This research builds on several recent advancements in this area, including transcriptomics-guided slide representation learning, morphological prototyping for unsupervised slide representation learning, generalizable whole slide image classification, hierarchical image pyramid transformer for colon cancer classification, and methods for finding regions of interest in whole slide images. These works demonstrate the potential of computational pathology to assist pathologists in tasks like diagnosis, prognosis, and treatment planning.
Plain English Explanation
• Computational pathology is a field that uses machine learning and AI to analyze medical images, such as those from whole slide scans of tissue samples.
• Recent research in this area has explored techniques like using gene expression data to learn image representations, unsupervised methods for learning image features, and approaches for classifying whole slide images and identifying important regions within them.
• These advancements show how computational pathology could help pathologists in their work, such as making diagnoses, predicting patient outcomes, and deciding on treatments.
Technical Explanation
• The paper builds on prior work in computational pathology, including techniques for transcriptomics-guided slide representation learning, morphological prototyping for unsupervised slide representation learning, generalizable whole slide image classification, hierarchical image pyramid transformer for colon cancer classification, and methods for finding regions of interest in whole slide images.
• These prior studies have demonstrated the potential of computational pathology techniques to assist pathologists in tasks like diagnosis, prognosis, and treatment planning by leveraging machine learning and AI to analyze histopathological images.
Critical Analysis
• The paper does not address certain limitations of the proposed self-supervised framework, such as its performance on rare or challenging disease cases, or its ability to generalize to diverse patient populations and tissue types.
• Further research is needed to validate the framework's clinical utility and compare it to pathologist-level performance on real-world diagnostic tasks.
• Additional work could explore ways to better incorporate domain knowledge from pathologists into the self-supervised learning process to further improve the model's interpretability and trustworthiness.
Conclusion
• This paper presents a self-supervised framework for learning representations of whole slide images, building on prior advancements in computational pathology.
• The proposed approach has the potential to assist pathologists in their work, but further research is needed to address limitations and validate its clinical utility.
• Overall, this work contributes to the growing field of computational pathology and its promise to enhance medical diagnosis and decision-making through the power of machine learning and AI.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
A self-supervised framework for learning whole slide representations
Xinhai Hou, Cheng Jiang, Akhil Kondepudi, Yiwei Lyu, Asadur Chowdury, Honglak Lee, Todd C. Hollon
Whole slide imaging is fundamental to biomedical microscopy and computational pathology. Previously, learning representations for gigapixel-sized whole slide images (WSIs) has relied on multiple instance learning with weak labels, which do not annotate the diverse morphologic features and spatial heterogeneity of WSIs. A high-quality self-supervised learning method for WSIs would provide transferable visual representations for downstream computational pathology tasks, without the need for dense annotations. We present Slide Pre-trained Transformers (SPT) for gigapixel-scale self-supervision of WSIs. Treating WSI patches as tokens, SPT combines data transformation strategies from language and vision modeling into a general and unified framework to generate views of WSIs for self-supervised pretraining. SPT leverages the inherent regional heterogeneity, histologic feature variability, and information redundancy within WSIs to learn high-quality whole slide representations. We benchmark SPT visual representations on five diagnostic tasks across three biomedical microscopy datasets. SPT significantly outperforms baselines for histopathologic diagnosis, cancer subtyping, and genetic mutation prediction. Finally, we demonstrate that SPT consistently improves whole slide representations when using off-the-shelf, in-domain, and foundational patch encoders for whole slide multiple instance learning.
Read more5/27/2024
0
Transcriptomics-guided Slide Representation Learning in Computational Pathology
Guillaume Jaume, Lukas Oldenburg, Anurag Vaidya, Richard J. Chen, Drew F. K. Williamson, Thomas Peeters, Andrew H. Song, Faisal Mahmood
Self-supervised learning (SSL) has been successful in building patch embeddings of small histology images (e.g., 224x224 pixels), but scaling these models to learn slide embeddings from the entirety of giga-pixel whole-slide images (WSIs) remains challenging. Here, we leverage complementary information from gene expression profiles to guide slide representation learning using multimodal pre-training. Expression profiles constitute highly detailed molecular descriptions of a tissue that we hypothesize offer a strong task-agnostic training signal for learning slide embeddings. Our slide and expression (S+E) pre-training strategy, called Tangle, employs modality-specific encoders, the outputs of which are aligned via contrastive learning. Tangle was pre-trained on samples from three different organs: liver (n=6,597 S+E pairs), breast (n=1,020), and lung (n=1,012) from two different species (Homo sapiens and Rattus norvegicus). Across three independent test datasets consisting of 1,265 breast WSIs, 1,946 lung WSIs, and 4,584 liver WSIs, Tangle shows significantly better few-shot performance compared to supervised and SSL baselines. When assessed using prototype-based classification and slide retrieval, Tangle also shows a substantial performance improvement over all baselines. Code available at https://github.com/mahmoodlab/TANGLE.
Read more5/21/2024
0
Multistain Pretraining for Slide Representation Learning in Pathology
Guillaume Jaume, Anurag Vaidya, Andrew Zhang, Andrew H. Song, Richard J. Chen, Sharifa Sahai, Dandan Mo, Emilio Madrigal, Long Phi Le, Faisal Mahmood
Developing self-supervised learning (SSL) models that can learn universal and transferable representations of H&E gigapixel whole-slide images (WSIs) is becoming increasingly valuable in computational pathology. These models hold the potential to advance critical tasks such as few-shot classification, slide retrieval, and patient stratification. Existing approaches for slide representation learning extend the principles of SSL from small images (e.g., 224 x 224 patches) to entire slides, usually by aligning two different augmentations (or views) of the slide. Yet the resulting representation remains constrained by the limited clinical and biological diversity of the views. Instead, we postulate that slides stained with multiple markers, such as immunohistochemistry, can be used as different views to form a rich task-agnostic training signal. To this end, we introduce Madeleine, a multimodal pretraining strategy for slide representation learning. Madeleine is trained with a dual global-local cross-stain alignment objective on large cohorts of breast cancer samples (N=4,211 WSIs across five stains) and kidney transplant samples (N=12,070 WSIs across four stains). We demonstrate the quality of slide representations learned by Madeleine on various downstream evaluations, ranging from morphological and molecular classification to prognostic prediction, comprising 21 tasks using 7,299 WSIs from multiple medical centers. Code is available at https://github.com/mahmoodlab/MADELEINE.
Read more8/7/2024
0
Morphological Prototyping for Unsupervised Slide Representation Learning in Computational Pathology
Andrew H. Song, Richard J. Chen, Tong Ding, Drew F. K. Williamson, Guillaume Jaume, Faisal Mahmood
Representation learning of pathology whole-slide images (WSIs) has been has primarily relied on weak supervision with Multiple Instance Learning (MIL). However, the slide representations resulting from this approach are highly tailored to specific clinical tasks, which limits their expressivity and generalization, particularly in scenarios with limited data. Instead, we hypothesize that morphological redundancy in tissue can be leveraged to build a task-agnostic slide representation in an unsupervised fashion. To this end, we introduce PANTHER, a prototype-based approach rooted in the Gaussian mixture model that summarizes the set of WSI patches into a much smaller set of morphological prototypes. Specifically, each patch is assumed to have been generated from a mixture distribution, where each mixture component represents a morphological exemplar. Utilizing the estimated mixture parameters, we then construct a compact slide representation that can be readily used for a wide range of downstream tasks. By performing an extensive evaluation of PANTHER on subtyping and survival tasks using 13 datasets, we show that 1) PANTHER outperforms or is on par with supervised MIL baselines and 2) the analysis of morphological prototypes brings new qualitative and quantitative insights into model interpretability.
Read more5/21/2024