Structure from Motion for Oblique Aerial Imagery

Master Thesis at ifp - Philipp Hüttl

Philipp Hüttl

Structure from Motion for Oblique Aerial Imagery

Duration of the Thesis: 6 months
Completion: November 2017
Supervisors: Dr.-Ing. Konrad Wenzel (nFrames), apl. Prof. Dr.-Ing. Norbert Haala
Examiner: apl. Prof. Dr.-Ing. Norbert Haala


 

Introduction

Oblique aerial images pose challenges for Structure from Motion (SfM) algorithms since they contain smaller image overlaps and larger perspective distortions than typical computer vision scenarios, leading to a lower redundancy [1][2]. This thesis therefore examines how these special characteristics affect the SfM workflow in the case of the open-source library Theia [3].

Methodology

Various tools for analysing the SfM workflow are utilized, each of them expressing information on a specific level of abstraction. Graph visualisations with Gephi [4] provide a global representation of the image pairs’ connectivity, see Figure 1. Especially when applying a force-based algorithm, the qualitative visualisations offer an intuitive way for identifying spatial differences. Variations in the relative accuracy of single image pairs are analysed in object space. Therefore dense point clouds are computed by triangulating both a nadir as well as an oblique stereo pair. The distances between these resulting point clouds increase for less accurate relative camera poses.

Figure 1: Georeferenced graph visualisation of a test dataset. Circles denote the camera positions, edge weights denote the image pair’s number of common tracks.

Analysis

With the above mentioned methods an exhaustive parameter evaluation is performed in order to find parameter values that are benefitial for reconstructing oblique aerial images. During these evaluations, highly overlapping datasets do not show large dependencies on the different parameters. Due to their high redundancy they can be accurately reconstructed even if many object points are filtered. However datasets with lower overlaps do not yield acceptable results for parameters far off the default values. For these datasets, a large number of feature matches, particularly between different viewing directions, is the most important requirement for obtaining accurate reconstructions. The correlation of the bundle’s connectivity and the obtained cloud-to-cloud distances is shown in Figures 2 and 3.

Figure 2: Flow Chart for Methodology
Figure 3: Cloud-to-cloud distances of the three image pairs highlighted in the force-based graph visualisation. The solution obtained with Theia is compared to reference data. Smaller Distances indicate a more accurate relative pose estimation.

Improvements

Establishing more point correspondences across viewing directions fails because common feature descriptors are not invariant to large perspective distortions. In order to overcome this restriction, a new method of image rectification is proposed. This method projects all images on a principal plane through the camera sensors. The images’ perspective distortions are artificially inverted for planar objects being parallel to the principal plane. The application of this method significantly increases the number of common tracks between different viewing directions. Furthermore, for lowly overlapping datasets, the relative accuracy in object space is improved, see Figure 4. After rectifying the images, the obtained cloud-to-cloud distances are in the same order of magnitude as the referenca data obtained by manually supervised aerotriangulation softwares. The largest advantage of this method is that no direct matches need to be found between different image pairs as long as all images are connected to the bundle. Another advantage is that each image needs to be transformed only once while the resampling can even be avoided for certain images.

Figure 4: Interactive Drawing

Conclusion

These results show that an accurate reconstruction can be obtained by using state of the art SfM algorithms as long as the input data features a high enough redundancy. For oblique images the bundle’s redundancy can be significantly increased by rectifying the input images with the proposed method.

References

[1] E. Rupnik, F. Nex, and F. Remondino. Oblique multi-camera systems - orientation and dense matching issues. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 40(3):107, 2014.

[2] K. Jacobsen. Geometry of vertical and oblique image combinations. In Remote Sensing for a Changing Europe: Proceedings of the 28th Symposium of the European Association of Remote Sensing Laboratories, Istanbul, Turkey, 2-5 June 2008, page 16. IOS Press, 2009.

[3] C. Sweeney, T. Hollerer, and M. Turk. Theia: A fast and scalable structure-from- motion library. In Proceedings of the 23rd ACM international conference on Multimedia, pages 693–696. ACM, 2015.

[4] M. Bastian, S. Heymann, and M. Jacomy. Gephi: An open source software for
exploring and manipulating networks, 2009.

Ansprechpartner

Dieses Bild zeigt Norbert Haala

Norbert Haala

apl. Prof. Dr.-Ing.

Stellvertretender Institutsleiter

Zum Seitenanfang