Fusion of Range Data from Multi-View Stereo and LiDAR for Volumetric Surface Reconstruction

Master Thesis at ifp - Yifan Song

Yifan Song

Fusion of Range Data from Multi-View Stereo and LiDAR for Volumetric Surface Reconstruction

Duration of the Thesis: 6 months
Completion: January 2017
Supervisor: Dipl.-Ing. Alessandro Cefalu
Examiner: Prof. Dr.-Ing. Norbert Haala


 

Introduction

Volumetric reconstruction of 3D surface models from multiple range images, as a very popular approach in 3D reconstruction scene, has occupied researchers’ attention for more than 20 years. Nowadays, emerging technologies like LiDAR system brought more alternatives and multiple measurements to the digital 3D reconstruction scheme. This thesis will focus on the volumetric surface reconstruction by fusing range images from Multi-view stereo (MVS) and Terrestrial Laser Scanner (TLS) data.

Data Processing

The workflow in this thesis follows as: Firstly, we use our program which programmed in C++ to convert the TLS data into to panoramic images (Color and Depth). Afterwards, we apply our correction model to correct the color panoramic images. Then all the color images (frame and panoramic) are loaded into the software Agisoft PhotoScan which aligns the images under the hybrid image matching configuration. The orientation parameters are computed for all images. Then after, software SURE is applied to generate the disparities for frame camera images by using SGM algorithm. Hereby, all the input data (MVS & TLS range images and corresponding orientation files) for the reconstruction pipeline are provided.

In this thesis, we implement the volumetric surface reconstruction in C++ under 3 different pipelines: (1) Linear Accumulation approach (2) Bayesian Accumulation of Gaussian (3) Total Variation (TV) Optimization.

Figure 1: Reconstruction details of St. Andrews Cathedral, Scotland based on (a) Linear Accumulation (b) Bayesian Accumulation of Gaussian (c) TV Optimization

For real-time application, Bayesian Accumulation of Gaussian approach is a good option. However, there are still many fragments in the corresponding surface model. Otherwise, The TV Optimization is recommended since it can provide a much cleaner and water-tight surface model.

 

Demonstration in real project dataset

A real project data was used to demonstrate MVS and LiDAR range image fusion for volumetric surface reconstruction under the TV Optimization pipeline. A sensor-weighted fusion approach was launched to further improve the fusion result.

Figure 2: Reconstructed interest of areas (red rectangles)
Figure 3: Surface Model of Object 1 which was reconstructed from: (a) Only MVS Range Images (b) Only TLS Panoramic Range Image (c) Unweighted Fusion of MVS and TLS Range Images (d) Sensor-Weighted Fusion of MVS and TLS Range Images
Figure 4: Surface Model of Object 2 which was reconstructed from: (a) Only MVS Range Images (b) Only TLS Panoramic Range Image (c) Unweighted Fusion of MVS and TLS Range Images (d) Sensor-Weighted Fusion of MVS and TLS Range Images

Conclusions

This thesis presented the volumetric based fusion approach for fusing range data from MVS and LiDAR. This fusion doesn’t need any extra sensor to provide orientation information as it can be directly computed by the Hybrid Image Matching algorithm. The fusion of different sources data significantly improved the quality of the surface model. However, several problems still exist in this thesis: (1) In TLS panoramic image generation part, the processing only considers the angular systematic errors, not rangefinder errors. (2)The panoramic color image correction model doesn’t have any physical interpretation. (3)The Scalability of the reconstruction is constrained by the memory consumption of the voxel field.

Ansprechpartner

Dieses Bild zeigt Norbert Haala
apl. Prof. Dr.-Ing.

Norbert Haala

Stellvertretender Institutsleiter

Zum Seitenanfang