GaSpCT: Gaussian Splatting for Novel CT Projection View Synthesis

Read original: arXiv:2404.03126 - Published 4/5/2024 by Emmanouil Nikolakakis, Utkarsh Gupta, Jonathan Vengosh, Justin Bui, Razvan Marinescu
Total Score

0

GaSpCT: Gaussian Splatting for Novel CT Projection View Synthesis

Sign in to get full access

or

If you already have an account, we'll log you in

Overview

  • The paper presents a novel method called GaSpCT (Gaussian Splatting for Novel CT Projection View Synthesis) for generating synthetic X-ray projection views from a limited set of acquired CT scans.
  • The proposed technique uses Gaussian splatting, a rendering technique that represents 3D data as a collection of Gaussian functions, to efficiently generate novel CT projection views.
  • The method aims to improve the quality of sparse-view CT reconstruction, which is important for reducing radiation exposure in medical imaging and enabling new applications.

Plain English Explanation

Computed tomography (CT) scans are a widely used medical imaging technology that produces 3D images of the body by taking a series of X-ray images from different angles. However, acquiring a full set of CT projection views can be time-consuming and expose patients to higher levels of radiation.

The GaSpCT: Gaussian Splatting for Novel CT Projection View Synthesis paper presents a new method to generate synthetic X-ray projection views from a limited set of acquired CT scans. This technique, known as Gaussian splatting, represents the 3D data as a collection of Gaussian functions, which can be efficiently rendered to produce novel projection views.

By using Gaussian splatting, the GaSpCT method can generate high-quality synthetic CT projection views without the need for a full set of acquired data. This can help reduce the radiation exposure to patients and enable new applications that require sparse-view CT reconstruction, such as real-time imaging during medical procedures.

The key idea behind Gaussian splatting is to model the 3D data as a collection of Gaussian functions, which can be easily integrated to produce the desired projection views. This approach is more efficient and flexible than traditional methods that rely on discrete voxel representations or complex reconstruction algorithms.

Technical Explanation

The GaSpCT: Gaussian Splatting for Novel CT Projection View Synthesis paper presents a novel method for generating synthetic X-ray projection views from a limited set of acquired CT scans using Gaussian splatting.

The core of the GaSpCT approach is to represent the 3D CT data as a collection of Gaussian functions, which can be efficiently integrated to produce the desired projection views. This is in contrast to traditional methods that rely on discrete voxel representations or complex reconstruction algorithms.

The paper first describes the Gaussian splatting technique, which models each 3D data point as a Gaussian function with a specified variance. These Gaussian splats are then integrated along the projection direction to generate the final synthetic projection view.

To enable efficient Gaussian splatting, the authors propose several techniques, including:

  1. Analytic Splatting: A method for analytically integrating the Gaussian splats to produce anti-aliased projection views.
  2. Instantsplat: An algorithm for efficiently splatting Gaussian functions onto a 2D projection plane.

The paper then demonstrates how the GaSpCT method can be used to generate high-quality synthetic CT projection views from a limited set of acquired data. The authors show that this approach can improve the quality of sparse-view CT reconstruction, which is important for reducing radiation exposure in medical imaging and enabling new applications.

Critical Analysis

The GaSpCT: Gaussian Splatting for Novel CT Projection View Synthesis paper presents a promising approach for generating synthetic CT projection views using Gaussian splatting. The key strengths of the method are its efficiency, flexibility, and potential to improve sparse-view CT reconstruction.

One potential limitation of the GaSpCT approach is that it relies on the assumption that the 3D data can be accurately represented as a collection of Gaussian functions. While this assumption may hold for some types of data, it may not be suitable for all CT applications, particularly those involving complex anatomical structures or materials with sharp boundaries.

Additionally, the paper does not provide a thorough analysis of the limitations of the Gaussian splatting technique, such as its sensitivity to the choice of Gaussian variance or potential issues with artifacts or distortions in the generated projection views.

Further research could explore the robustness of the GaSpCT method to different types of CT data, as well as investigate ways to extend the technique to handle more complex data representations or incorporate additional constraints or priors to improve the quality of the synthetic projection views.

Conclusion

The GaSpCT: Gaussian Splatting for Novel CT Projection View Synthesis paper presents a novel method for efficiently generating synthetic X-ray projection views from a limited set of acquired CT scans. By using Gaussian splatting, the GaSpCT approach can produce high-quality synthetic projection views without the need for a full set of acquired data, which can help reduce radiation exposure in medical imaging and enable new applications that require sparse-view CT reconstruction.

The key strengths of the GaSpCT method are its efficiency, flexibility, and potential to improve the quality of sparse-view CT reconstruction. While the paper demonstrates promising results, further research is needed to explore the limitations of the Gaussian splatting technique and investigate ways to extend the method to handle more complex data representations or incorporate additional constraints to improve the quality of the synthetic projection views.

Overall, the GaSpCT paper represents an important contribution to the field of medical imaging, with the potential to significantly impact the way CT scans are acquired and processed in the future.



This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

GaSpCT: Gaussian Splatting for Novel CT Projection View Synthesis
Total Score

0

GaSpCT: Gaussian Splatting for Novel CT Projection View Synthesis

Emmanouil Nikolakakis, Utkarsh Gupta, Jonathan Vengosh, Justin Bui, Razvan Marinescu

We present GaSpCT, a novel view synthesis and 3D scene representation method used to generate novel projection views for Computer Tomography (CT) scans. We adapt the Gaussian Splatting framework to enable novel view synthesis in CT based on limited sets of 2D image projections and without the need for Structure from Motion (SfM) methodologies. Therefore, we reduce the total scanning duration and the amount of radiation dose the patient receives during the scan. We adapted the loss function to our use-case by encouraging a stronger background and foreground distinction using two sparsity promoting regularizers: a beta loss and a total variation (TV) loss. Finally, we initialize the Gaussian locations across the 3D space using a uniform prior distribution of where the brain's positioning would be expected to be within the field of view. We evaluate the performance of our model using brain CT scans from the Parkinson's Progression Markers Initiative (PPMI) dataset and demonstrate that the rendered novel views closely match the original projection views of the simulated scan, and have better performance than other implicit 3D scene representations methodologies. Furthermore, we empirically observe reduced training time compared to neural network based image synthesis for sparse-view CT image reconstruction. Finally, the memory requirements of the Gaussian Splatting representations are reduced by 17% compared to the equivalent voxel grid image representations.

Read more

4/5/2024

🏷️

Total Score

0

GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis

Shunyuan Zheng, Boyao Zhou, Ruizhi Shao, Boning Liu, Shengping Zhang, Liqiang Nie, Yebin Liu

We present a new approach, termed GPS-Gaussian, for synthesizing novel views of a character in a real-time manner. The proposed method enables 2K-resolution rendering under a sparse-view camera setting. Unlike the original Gaussian Splatting or neural implicit rendering methods that necessitate per-subject optimizations, we introduce Gaussian parameter maps defined on the source views and regress directly Gaussian Splatting properties for instant novel view synthesis without any fine-tuning or optimization. To this end, we train our Gaussian parameter regression module on a large amount of human scan data, jointly with a depth estimation module to lift 2D parameter maps to 3D space. The proposed framework is fully differentiable and experiments on several datasets demonstrate that our method outperforms state-of-the-art methods while achieving an exceeding rendering speed.

Read more

4/17/2024

📉

Total Score

0

Gaussian Splatting: 3D Reconstruction and Novel View Synthesis, a Review

Anurag Dalal, Daniel Hagen, Kjell G. Robbersmyr, Kristian Muri Knausg{aa}rd

Image-based 3D reconstruction is a challenging task that involves inferring the 3D shape of an object or scene from a set of input images. Learning-based methods have gained attention for their ability to directly estimate 3D shapes. This review paper focuses on state-of-the-art techniques for 3D reconstruction, including the generation of novel, unseen views. An overview of recent developments in the Gaussian Splatting method is provided, covering input types, model structures, output representations, and training strategies. Unresolved challenges and future directions are also discussed. Given the rapid progress in this domain and the numerous opportunities for enhancing 3D reconstruction methods, a comprehensive examination of algorithms appears essential. Consequently, this study offers a thorough overview of the latest advancements in Gaussian Splatting.

Read more

5/7/2024

Feature Splatting for Better Novel View Synthesis with Low Overlap
Total Score

0

Feature Splatting for Better Novel View Synthesis with Low Overlap

T. Berriel Martins, Javier Civera

3D Gaussian Splatting has emerged as a very promising scene representation, achieving state-of-the-art quality in novel view synthesis significantly faster than competing alternatives. However, its use of spherical harmonics to represent scene colors limits the expressivity of 3D Gaussians and, as a consequence, the capability of the representation to generalize as we move away from the training views. In this paper, we propose to encode the color information of 3D Gaussians into per-Gaussian feature vectors, which we denote as Feature Splatting (FeatSplat). To synthesize a novel view, Gaussians are first splatted into the image plane, then the corresponding feature vectors are alpha-blended, and finally the blended vector is decoded by a small MLP to render the RGB pixel values. To further inform the model, we concatenate a camera embedding to the blended feature vector, to condition the decoding also on the viewpoint information. Our experiments show that these novel model for encoding the radiance considerably improves novel view synthesis for low overlap views that are distant from the training views. Finally, we also show the capacity and convenience of our feature vector representation, demonstrating its capability not only to generate RGB values for novel views, but also their per-pixel semantic labels. We will release the code upon acceptance. Keywords: Gaussian Splatting, Novel View Synthesis, Feature Splatting

Read more

5/27/2024