Open-source High-precision Autonomous Suturing Framework With Visual Guidance

Read original: arXiv:2210.01406 - Published 5/29/2024 by Hongbin Lin, Bin Li, Yunhui Liu, Kwok Wai Samuel Au
Total Score

0

๐Ÿงช

Sign in to get full access

or

If you already have an account, we'll log you in

Overview

  • Autonomous surgery is an emerging field aiming to revolutionize robotic patient care, but remains a challenging goal.
  • This paper proposes an image-based framework for high-precision autonomous suturing operations.
  • The framework includes an algorithm for accurate needle pose estimation, a calibration network for compensating joint offsets, and suture trajectory planning and control.
  • The proposed solution ranked first in the AccelNet Surgical Robotics Challenge.

Plain English Explanation

Autonomous surgery, where robots perform medical procedures without human intervention, is an exciting but difficult goal. This paper describes a new system that aims to bring us closer to that reality. The key components are:

  1. An algorithm that can accurately estimate the position and orientation of a surgical needle. This is crucial for the robot to know where the needle is during the procedure.

  2. A calibration network that compensates for small differences between the robot's joints and the actual movement, ensuring the robot's actions match the planned trajectory.

  3. Precise planning and control of the suture (stitching) trajectory, allowing the robot to carefully guide the needle and thread through the tissue.

The researchers tested this system in a surgical robotics competition and their solution came out on top, demonstrating the potential of this approach for autonomous suturing tasks.

Technical Explanation

The paper proposes an image-based framework for autonomous suturing operations, consisting of three key components:

  1. Algebraic geometric needle pose estimation: The researchers developed an algorithm to accurately determine the 3D position and orientation of the surgical needle based on camera images. This is a crucial first step for the robot to know where the needle is during the procedure.

  2. Keypoint-based calibration network: To compensate for small discrepancies between the robot's joint movements and the actual motion of the end-effector, the authors designed a calibration network that uses key points on the robot to learn and correct these offsets.

  3. Suture trajectory planning and control: Based on the needle pose and calibrated robot model, the framework plans and executes the optimal trajectory for the suturing operation, guiding the needle and thread through the tissue.

The researchers evaluated their framework in the AccelNet Surgical Robotics Challenge, where it achieved the top ranking among all competitors.

Critical Analysis

The paper presents a promising approach for autonomous suturing, but also acknowledges several limitations and areas for further research:

  • The needle pose estimation and calibration were evaluated in controlled laboratory settings, and the authors note that real-world surgical environments may introduce additional challenges.
  • The suturing task was demonstrated on a simple planar surface, whereas actual surgical procedures often involve complex 3D tissue geometries and deformations. Extensions to more realistic patient-specific scenarios would be an important next step.
  • The paper focuses on suturing, but autonomous surgery requires addressing a broader set of tasks, such as tissue manipulation and human-robot collaboration.

Overall, the research represents a solid step towards autonomous surgical capabilities, but significant challenges remain before such systems can be safely deployed in real clinical settings.

Conclusion

This paper presents a novel image-based framework for autonomous suturing operations, addressing key technical challenges such as accurate needle pose estimation, robot calibration, and trajectory planning. The system's strong performance in a surgical robotics competition demonstrates its potential to advance the field of autonomous surgery. However, the authors acknowledge limitations around real-world deployment, highlighting the need for further research to tackle the complexity of actual surgical environments and procedures. As autonomous surgery continues to evolve, this work contributes important insights and capabilities towards the eventual goal of revolutionizing robotic patient care.



This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on ๐• โ†’

Related Papers

๐Ÿงช

Total Score

0

Open-source High-precision Autonomous Suturing Framework With Visual Guidance

Hongbin Lin, Bin Li, Yunhui Liu, Kwok Wai Samuel Au

Autonomous surgery has attracted increasing attention for revolutionizing robotic patient care, yet remains a distant and challenging goal. In this paper, we propose an image-based framework for high-precision autonomous suturing operation. We first build an algebraic geometric algorithm to achieve accurate needle pose estimation, then design the corresponding keypoint-based calibration network for joint-offset compensation, and further plan and control suture trajectory. Our solution ranked first among all competitors in the AccelNet Surgical Robotics Challenge. Videos and codes can be found in https://sites.google.com/view/accel-2022-cuhk.

Read more

5/29/2024

Autonomous Image-to-Grasp Robotic Suturing Using Reliability-Driven Suture Thread Reconstruction
Total Score

0

Autonomous Image-to-Grasp Robotic Suturing Using Reliability-Driven Suture Thread Reconstruction

Neelay Joglekar, Fei Liu, Florian Richter, Michael C. Yip

Automating suturing during robotically-assisted surgery reduces the burden on the operating surgeon, enabling them to focus on making higher-level decisions rather than fatiguing themselves in the numerous intricacies of a surgical procedure. Accurate suture thread reconstruction and grasping are vital prerequisites for suturing, particularly for avoiding entanglement with surgical tools and performing complex thread manipulation. However, such methods must be robust to heavy perceptual degradation resulting from heavy noise and thread feature sparsity from endoscopic images. We develop a reconstruction algorithm that utilizes quadratic programming optimization to fit smooth splines to thread observations, satisfying reliability bounds estimated from measured observation noise. Additionally, we craft a grasping policy that generates gripper trajectories that maximize the probability of a successful grasp. Our full image-to-grasp pipeline is rigorously evaluated with over 400 grasping trials, exhibiting state-of-the-art accuracy. We show that this strategy can be applied to the various techniques in autonomous suture needle manipulation to achieve autonomous surgery in a generalizable way.

Read more

9/2/2024

Suturing Tasks Automation Based on Skills Learned From Demonstrations: A Simulation Study
Total Score

0

Suturing Tasks Automation Based on Skills Learned From Demonstrations: A Simulation Study

Haoying Zhou, Yiwei Jiang, Shang Gao, Shiyue Wang, Peter Kazanzides, Gregory S. Fischer

In this work, we develop an open-source surgical simulation environment that includes a realistic model obtained by MRI-scanning a physical phantom, for the purpose of training and evaluating a Learning from Demonstration (LfD) algorithm for autonomous suturing. The LfD algorithm utilizes Dynamic Movement Primitives (DMP) and Locally Weighted Regression (LWR), but focuses on the needle trajectory, rather than the instruments, to obtain better generality with respect to needle grasps. We conduct a user study to collect multiple suturing demonstrations and perform a comprehensive analysis of the ability of the LfD algorithm to generalize from a demonstration at one location in one phantom to different locations in the same phantom and to a different phantom. Our results indicate good generalization, on the order of 91.5%, when learning from more experienced subjects, indicating the need to integrate skill assessment in the future.

Read more

6/6/2024

STITCH: Augmented Dexterity for Suture Throws Including Thread Coordination and Handoffs
Total Score

0

STITCH: Augmented Dexterity for Suture Throws Including Thread Coordination and Handoffs

Kush Hari, Hansoul Kim, Will Panitch, Kishore Srinivas, Vincent Schorp, Karthik Dharmarajan, Shreya Ganti, Tara Sadjadpour, Ken Goldberg

We present STITCH: an augmented dexterity pipeline that performs Suture Throws Including Thread Coordination and Handoffs. STITCH iteratively performs needle insertion, thread sweeping, needle extraction, suture cinching, needle handover, and needle pose correction with failure recovery policies. We introduce a novel visual 6D needle pose estimation framework using a stereo camera pair and new suturing motion primitives. We compare STITCH to baselines, including a proprioception-only and a policy without visual servoing. In physical experiments across 15 trials, STITCH achieves an average of 2.93 sutures without human intervention and 4.47 sutures with human intervention. See https://sites.google.com/berkeley.edu/stitch for code and supplemental materials.

Read more

4/9/2024