Replacing Objects in Point Cloud stream with Real-time Meshes using Semantic Segmentation

Abstract: Background: The evolving landscape of 3D data processing, particularly pointcloud manipulation, is pivotal in numerous applications ranging from architecturaldesign to spatial analysis. Traditional methods, primarily mesh generation from point clouds, face challenges in adapting to complex real-world scenarios. This thesis addresses the need for improved methodologies in extracting and representing objects within point clouds, focusing on enhancing visual fidelity and practicality.Objectives: The primary objective of this research is to explore and compare two distinct methodologies for processing point cloud data: traditional mesh generation and the innovative approach of mesh replacement using pre-existing 3D meshes. The study aims to establish which method better achieves high visual accuracy and efficiency in reconstructing 3D environments from point clouds.Methods: This research employs a comparative analytical framework, wherein both mesh generation and replacement techniques are applied to identical pointcloud datasets. The mesh generation method involves semantic segmentation and direct mesh creation, while the mesh replacement technique utilises pre-selected,high-quality meshes from a library. The evaluation criteria focus on visual fidelity,accuracy, and practical applicability of each method.Results: The findings reveal that mesh replacement with pre-existing 3D mesh significantly enhances the visual quality of the reconstructed environments compared to traditional mesh generation. This method demonstrated a higher degree of accuracy in replicating object shapes and details, leading to more realistic and visually appealing models.Conclusions: The study concludes that replacing point cloud segments with preexisting meshes is a superior approach to traditional mesh generation in terms of visual fidelity and realism. This method has broad implications for various fields,potentially improving the quality and efficiency of 3D environment reconstruction.While the primary focus is not on VR/AR applications, the advancements proposed here bear significant potential for enhancing these technologies.Keywords: Point Cloud Processing, Mesh Generation, Mesh Replacement, 3D Modeling, Visual Fidelity