Zero-shot 3D Segmentation of Abdominal Organs in CT Scans Using Segment Anything Model 2: Adapting Video Tracking Capabilities for 3D Medical Imaging

Overview

• This study evaluated the zero-shot performance of the Segment Anything Model 2 (SAM 2) in segmenting abdominal organs from CT scans.
• The researchers used a subset of the TotalSegmentator CT dataset to assess SAM 2's ability to segment 8 different abdominal organs.
• Segmentation was initiated from three different Z-coordinate levels of each organ, and performance was measured using the Dice similarity coefficient (DSC).
• The researchers also analyzed organ volumes to provide context for the results.

Plain English Explanation

The researchers wanted to see how well the Segment Anything Model 2 (SAM 2) could segment, or outline, different organs in CT scans of the abdomen without any additional training. CT scans are 3D images of the body that doctors use to diagnose and monitor medical conditions.

The researchers used a dataset of 123 CT scans from 8 different hospitals to test SAM 2's performance on 8 abdominal organs: the liver, left and right kidneys, spleen, gallbladder, pancreas, and adrenal glands. They started the segmentation process at three different heights (or Z-coordinates) within the scans to see how that affected the results.

The researchers measured the accuracy of the segmentation using a metric called the Dice similarity coefficient (DSC). DSC values range from 0 to 1, with 1 indicating a perfect match between the model's segmentation and the ground truth.

The results showed that SAM 2 performed well on larger organs with clear boundaries, like the liver, kidneys, and spleen, achieving high DSC scores. However, it struggled more with smaller or less defined structures, like the gallbladder, pancreas, and adrenal glands.

The researchers also found that the starting point of the segmentation (the initial slice) made a significant difference in the results for different organs. Additionally, there was a lot of variability in the model's performance, with DSC values ranging from near 0 to almost 1 for the same organs across different scans.

Overall, the study demonstrates that SAM 2 has promising zero-shot capabilities for segmenting certain abdominal organs in CT scans, but it still has room for improvement, especially when it comes to smaller and less distinct structures.

Technical Explanation

The researchers used a zero-shot approach to evaluate the performance of the Segment Anything Model 2 (SAM 2) in segmenting 8 different abdominal organs from CT scans.

They leveraged a subset of the TotalSegmentator CT dataset (n=123) from 8 different institutions to assess SAM 2's ability to segment the liver, left kidney, right kidney, spleen, gallbladder, pancreas, and adrenal glands.

Segmentation was initiated from three different Z-coordinate levels (caudal, mid, and cranial) of each organ. Performance was measured using the Dice similarity coefficient (DSC), which quantifies the overlap between the model's segmentation and the ground truth. The researchers also analyzed organ volumes to provide context for the results.

The findings showed that larger organs with clear boundaries, such as the liver (mean(median) DSC: 0.821(0.898)), left kidney (0.870(0.921)), right kidney (0.862(0.935)), and spleen (0.891(0.932)), demonstrated high segmentation performance in the zero-shot setting.

However, smaller or less defined structures, like the gallbladder (0.531(0.590)), pancreas (0.361(0.359)), and adrenal glands (0.203-0.308(0.109-0.231)), exhibited lower performance. Significant differences in DSC were observed depending on the starting initial slice of segmentation for different organs.

A moderate positive correlation was found between organ volume and DSC (Spearman's rs = 0.731, P <.001 at the caudal level), indicating that larger organs were easier for the model to segment. The results also showed high variability in DSC values within organs, ranging from near 0 to almost 1.0, suggesting substantial inconsistency in segmentation performance between scans.

Critical Analysis

The study demonstrates the promising zero-shot capabilities of SAM 2 in segmenting certain abdominal organs from CT scans, particularly larger structures with clear boundaries. This highlights the model's potential for cross-domain generalization in medical imaging, where it can be applied to new tasks without the need for additional training.

However, the researchers also acknowledge the limitations of the model's performance, particularly for smaller and less defined organs. This suggests that further improvements are needed to enhance the model's ability to accurately segment a wider range of anatomical structures in medical imaging.

Additionally, the study's finding of significant differences in segmentation performance based on the starting slice of the segmentation process raises questions about the model's robustness and consistency across different scan regions. This variability in results may be an important consideration when deploying such models in clinical settings.

While the study provides valuable insights into the zero-shot capabilities of SAM 2, further research is needed to explore the model's applicability in more diverse medical imaging scenarios, as well as to investigate potential strategies for improving its performance on challenging anatomical structures.

Conclusion

This study evaluated the zero-shot performance of the Segment Anything Model 2 (SAM 2) in segmenting abdominal organs from CT scans. The results demonstrate that SAM 2 can effectively segment larger organs with clear boundaries, such as the liver, kidneys, and spleen, without any additional training.

However, the model struggled more with smaller or less defined structures, like the gallbladder, pancreas, and adrenal glands. The study also highlighted the importance of the starting point of the segmentation process and the substantial variability in the model's performance across different scans.

Overall, the findings suggest that SAM 2 has promising zero-shot capabilities for certain medical imaging tasks, but further improvements are needed to enhance its performance on a wider range of anatomical structures. This study provides valuable insights into the potential and limitations of this advanced segmentation model in the context of 3D medical imaging.