Towards Fine-grained Large Object Segmentation 1st Place Solution to 3D AI Challenge 2020 -- Instance Segmentation Track

Overview

• This technical report introduces a solution by Team 'FineGrainedSeg' for the Instance Segmentation track in the 3D AI Challenge 2020.
• To handle extremely large objects in 3D-FUTURE, the team adopted PointRend as their basic framework, which outputs more fine-grained masks compared to other methods like HTC and SOLOv2.
• The final submission is an ensemble of 5 PointRend models, which achieved the 1st place on both the validation and test leaderboards.

Plain English Explanation

The team's goal was to develop a solution for the 3D Instance Segmentation task in the 3D AI Challenge 2020. They recognized that dealing with very large objects in the 3D-FUTURE dataset was a key challenge. To address this, they used a model called PointRend, which can produce more detailed and precise object masks compared to other approaches. The final system they submitted was an ensemble of 5 PointRend models, which performed the best on both the validation and test sets, earning them the 1st place position.

Technical Explanation

The core of the team's approach was the PointRend model, which they chose as their basic framework. PointRend is a state-of-the-art instance segmentation model that can output more fine-grained object masks compared to other methods like Hybrid Task Cascade (HTC) and SOLOv2. This was crucial for handling the extremely large objects present in the 3D-FUTURE dataset.

To further improve performance, the team created an ensemble of 5 PointRend models, which allowed them to combine the strengths of multiple versions of the same architecture. This ensemble approach led to their solution achieving the top spot on both the validation and test leaderboards for the 3D Instance Segmentation task.

Critical Analysis

The paper does not provide a detailed analysis of the limitations or potential issues with the proposed solution. While the ensemble of PointRend models performed well, the researchers could have discussed the trade-offs or challenges in training and deploying such a complex system.

Additionally, the paper does not compare the team's approach to other state-of-the-art 3D instance segmentation methods, such as What is Point Supervision Worth for Video Instance Segmentation?, 3D Open Vocabulary Panoptic Segmentation, or iSeg: Interactive 3D Segmentation via Interactive Attention. Such a comparison could have provided a more comprehensive understanding of the strengths and weaknesses of the team's approach.

Conclusion

The team's solution for the 3D Instance Segmentation task in the 3D AI Challenge 2020 centered around the use of the PointRend model, which allowed them to generate more detailed object masks. By ensembling multiple PointRend models, they were able to achieve the top spot on both the validation and test leaderboards. This work demonstrates the potential of advanced instance segmentation techniques, such as PointRend, for handling challenging 3D data. However, the paper could have provided a more thorough analysis of the solution's limitations and a comparison to other state-of-the-art approaches.