Human-in-the-Loop Model-Driven Interactive Point Cloud Segmentation in VR

Abstract

Given the task of labeling and cleaning point cloud–based 3D data, existing solutions are typically either fully manual or fully automated. Manual labeling in 3D requires substantial time investment, while automated approaches often lack robustness and still demand post-processing and supervision to ensure quality results. We argue that both domains may benefit from an integrated approach and therefore propose an immersive VR tool for instance segmentation and labeling of 3D point clouds with model-assisted automation. The tool combines two state-of-the-art machine learning methods for instance segmentation with standard functionalities for manual cleaning and labeling.This hybrid design enables partial automation to reduce labeling effort while still allowing supervised corrections, ultimately creating a semi-automated system.