Fusion of Range Data from Multi-View Stereo and LiDAR for Volumetric Surface Reconstruction

Duration of the Thesis: 6 months
Completion: January 2017
Supervisor: Dipl.-Ing. Alessandro Cefalu
Examiner: Prof. Dr.-Ing. Norbert Haala

Volumetric reconstruction of 3D surface models from multiple range images, as a very popular approach in 3D reconstruction scene, has occupied researchers’ attention for more than 20 years. Nowadays, emerging technologies like LiDAR system brought more alternatives and multiple measurements to the digital 3D reconstruction scheme. This thesis will focus on the volumetric surface reconstruction by fusing range images from Multi-view stereo (MVS) and Terrestrial Laser Scanner (TLS) data.

The workflow in this thesis follows as: Firstly, we use our program which programmed in C++ to convert the TLS data into to panoramic images (Color and Depth). Afterwards, we apply our correction model to correct the color panoramic images. Then all the color images (frame and panoramic) are loaded into the software Agisoft PhotoScan which aligns the images under the hybrid image matching configuration. The orientation parameters are computed for all images. Then after, software SURE is applied to generate the disparities for frame camera images by using SGM algorithm. Hereby, all the input data (MVS & TLS range images and corresponding orientation files) for the reconstruction pipeline are provided.

In this thesis, we implement the volumetric surface reconstruction in C++ under 3 different pipelines: (1) Linear Accumulation approach (2) Bayesian Accumulation of Gaussian (3) Total Variation (TV) Optimization.

For real-time application, Bayesian Accumulation of Gaussian approach is a good option. However, there are still many fragments in the corresponding surface model. Otherwise, The TV Optimization is recommended since it can provide a much cleaner and water-tight surface model.

Demonstration in real project dataset

A real project data was used to demonstrate MVS and LiDAR range image fusion for volumetric surface reconstruction under the TV Optimization pipeline. A sensor-weighted fusion approach was launched to further improve the fusion result.

This thesis presented the volumetric based fusion approach for fusing range data from MVS and LiDAR. This fusion doesn’t need any extra sensor to provide orientation information as it can be directly computed by the Hybrid Image Matching algorithm. The fusion of different sources data significantly improved the quality of the surface model. However, several problems still exist in this thesis: (1) In TLS panoramic image generation part, the processing only considers the angular systematic errors, not rangefinder errors. (2)The panoramic color image correction model doesn’t have any physical interpretation. (3)The Scalability of the reconstruction is constrained by the memory consumption of the voxel field.