CholecInstanceSeg: A Tool Instance Segmentation Dataset for Laparoscopic Surgery
0
Sign in to get full access
Overview
- This paper introduces CholecInstanceSeg, a new dataset for tool instance segmentation in laparoscopic surgery videos.
- The dataset contains annotated images of surgical tools used during cholecystectomy (gallbladder removal) procedures.
- The goal is to enable improved computer vision-based detection and segmentation of surgical tools to assist surgeons during operations.
Plain English Explanation
CholecInstanceSeg is a new dataset that can help improve computer vision systems used in laparoscopic surgery. Laparoscopic surgery involves making small incisions and using a camera to perform operations inside the body, like removing the gallbladder.
The dataset contains many images of the different surgical tools used during these gallbladder removal procedures, with each tool individually labeled and segmented. This allows AI models to better recognize and understand the specific tools being used at any given time during a surgery.
Having a comprehensive dataset of annotated surgical tools can enable the development of more advanced computer vision algorithms. These algorithms could then be used to automatically detect and track the tools used by surgeons in real-time, potentially assisting them during the procedure. This could improve surgical outcomes by providing surgeons with better information and decision support.
Technical Explanation
The CholecInstanceSeg dataset consists of over 10,000 annotated frames from laparoscopic cholecystectomy videos. Each frame contains pixel-level segmentation masks for the various surgical tools present, such as graspers, scissors, and cautery instruments.
The dataset was created by manually annotating tool instances in a large collection of surgical video data using a semi-automatic tool segmentation pipeline. This allowed the researchers to efficiently generate high-quality instance-level annotations for a diverse set of tools across many different procedures.
Evaluation of state-of-the-art instance segmentation models on the CholecInstanceSeg dataset shows promising results, with the best models achieving over 80% mean average precision on the task. However, the paper also identifies several challenges, such as tool occlusion and varied lighting conditions, that limit current model performance.
The availability of CholecInstanceSeg provides a valuable new resource for developing and benchmarking computer vision techniques for surgical tool detection and tracking. This could lead to advancements in Comprehensive Robotic Cholecystectomy Dataset (CRCD), SurgiTrack, EgoSurgery, and other surgical AI systems.
Critical Analysis
The CholecInstanceSeg dataset represents an important step forward in providing high-quality, annotated data for surgical tool segmentation. The comprehensive annotations and diverse set of tools covered are valuable for training and evaluating computer vision models.
However, the paper acknowledges several limitations of the dataset. The annotations are based on a single-view, 2D video feed, which may not capture all the nuances of tool usage and interactions during 3D laparoscopic procedures. There is also a need for more diverse data, including different surgical procedures beyond just cholecystectomies.
Additionally, the authors note challenges in accurately segmenting tools due to issues like occlusion and varied lighting. These are known difficulties in surgical computer vision that will require further research and innovation to overcome. Techniques like ChexMask for anatomical segmentation may provide insights.
Overall, CholecInstanceSeg is a promising step, but continued advancements in surgical data collection, annotation, and model development will be necessary to realize the full potential of computer-assisted laparoscopic surgery. Approaches like rule-based outlier detection may also help identify and mitigate potential issues in AI-driven surgical systems.
Conclusion
The CholecInstanceSeg dataset provides a valuable new resource for advancing computer vision techniques in laparoscopic surgery. By enabling more accurate detection and segmentation of surgical tools, this dataset can contribute to the development of AI systems that can assist surgeons during complex procedures, potentially improving patient outcomes.
While the dataset has some limitations, it represents an important step forward in bridging the gap between computer vision and surgical robotics. Continued research and innovation in this area could lead to transformative advances in computer-assisted surgery, benefiting both healthcare professionals and the patients they serve.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
CholecInstanceSeg: A Tool Instance Segmentation Dataset for Laparoscopic Surgery
Oluwatosin Alabi, Ko Ko Zayar Toe, Zijian Zhou, Charlie Budd, Nicholas Raison, Miaojing Shi, Tom Vercauteren
In laparoscopic and robotic surgery, precise tool instance segmentation is an essential technology for advanced computer-assisted interventions. Although publicly available procedures of routine surgeries exist, they often lack comprehensive annotations for tool instance segmentation. Additionally, the majority of standard datasets for tool segmentation are derived from porcine(pig) surgeries. To address this gap, we introduce CholecInstanceSeg, the largest open-access tool instance segmentation dataset to date. Derived from the existing CholecT50 and Cholec80 datasets, CholecInstanceSeg provides novel annotations for laparoscopic cholecystectomy procedures in patients. Our dataset comprises 41.9k annotated frames extracted from 85 clinical procedures and 64.4k tool instances, each labelled with semantic masks and instance IDs. To ensure the reliability of our annotations, we perform extensive quality control, conduct label agreement statistics, and benchmark the segmentation results with various instance segmentation baselines. CholecInstanceSeg aims to advance the field by offering a comprehensive and high-quality open-access dataset for the development and evaluation of tool instance segmentation algorithms.
Read more6/26/2024
🤷
0
Comprehensive Robotic Cholecystectomy Dataset (CRCD): Integrating Kinematics, Pedal Signals, and Endoscopic Videos
Ki-Hwan Oh, Leonardo Borgioli, Alberto Mangano, Valentina Valle, Marco Di Pangrazio, Francesco Toti, Gioia Pozza, Luciano Ambrosini, Alvaro Ducas, Milos Zefran, Liaohai Chen, Pier Cristoforo Giulianotti
In recent years, the potential applications of machine learning to Minimally Invasive Surgery (MIS) have spurred interest in data sets that can be used to develop data-driven tools. This paper introduces a novel dataset recorded during ex vivo pseudo-cholecystectomy procedures on pig livers, utilizing the da Vinci Research Kit (dVRK). Unlike current datasets, ours bridges a critical gap by offering not only full kinematic data but also capturing all pedal inputs used during the procedure and providing a time-stamped record of the endoscope's movements. Contributed by seven surgeons, this data set introduces a new dimension to surgical robotics research, allowing the creation of advanced models for automating console functionalities. Our work addresses the existing limitation of incomplete recordings and imprecise kinematic data, common in other datasets. By introducing two models, dedicated to predicting clutch usage and camera activation, we highlight the dataset's potential for advancing automation in surgical robotics. The comparison of methodologies and time windows provides insights into the models' boundaries and limitations.
Read more4/9/2024
0
SurgiTrack: Fine-Grained Multi-Class Multi-Tool Tracking in Surgical Videos
Chinedu Innocent Nwoye, Nicolas Padoy
Accurate tool tracking is essential for the success of computer-assisted intervention. Previous efforts often modeled tool trajectories rigidly, overlooking the dynamic nature of surgical procedures, especially tracking scenarios like out-of-body and out-of-camera views. Addressing this limitation, the new CholecTrack20 dataset provides detailed labels that account for multiple tool trajectories in three perspectives: (1) intraoperative, (2) intracorporeal, and (3) visibility, representing the different types of temporal duration of tool tracks. These fine-grained labels enhance tracking flexibility but also increase the task complexity. Re-identifying tools after occlusion or re-insertion into the body remains challenging due to high visual similarity, especially among tools of the same category. This work recognizes the critical role of the tool operators in distinguishing tool track instances, especially those belonging to the same tool category. The operators' information are however not explicitly captured in surgical videos. We therefore propose SurgiTrack, a novel deep learning method that leverages YOLOv7 for precise tool detection and employs an attention mechanism to model the originating direction of the tools, as a proxy to their operators, for tool re-identification. To handle diverse tool trajectory perspectives, SurgiTrack employs a harmonizing bipartite matching graph, minimizing conflicts and ensuring accurate tool identity association. Experimental results on CholecTrack20 demonstrate SurgiTrack's effectiveness, outperforming baselines and state-of-the-art methods with real-time inference capability. This work sets a new standard in surgical tool tracking, providing dynamic trajectories for more adaptable and precise assistance in minimally invasive surgeries.
Read more5/31/2024
0
EgoSurgery-Tool: A Dataset of Surgical Tool and Hand Detection from Egocentric Open Surgery Videos
Ryo Fujii, Hideo Saito, Hiroki Kajita
Surgical tool detection is a fundamental task for understanding egocentric open surgery videos. However, detecting surgical tools presents significant challenges due to their highly imbalanced class distribution, similar shapes and similar textures, and heavy occlusion. The lack of a comprehensive large-scale dataset compounds these challenges. In this paper, we introduce EgoSurgery-Tool, an extension of the existing EgoSurgery-Phase dataset, which contains real open surgery videos captured using an egocentric camera attached to the surgeon's head, along with phase annotations. EgoSurgery-Tool has been densely annotated with surgical tools and comprises over 49K surgical tool bounding boxes across 15 categories, constituting a large-scale surgical tool detection dataset. EgoSurgery-Tool also provides annotations for hand detection with over 46K hand-bounding boxes, capturing hand-object interactions that are crucial for understanding activities in egocentric open surgery. EgoSurgery-Tool is superior to existing datasets due to its larger scale, greater variety of surgical tools, more annotations, and denser scenes. We conduct a comprehensive analysis of EgoSurgery-Tool using nine popular object detectors to assess their effectiveness in both surgical tool and hand detection. The dataset will be released at https://github.com/Fujiry0/EgoSurgery.
Read more6/7/2024