Multi-Modal Object Pose Estimation with a Graph-Based System for Gangway Operation

Duration of the Thesis: 6 months
Completion: October 2019
Supervisor: Nguyen Quang Huy
Examiner: Prof. Dr.-Ing. Norbert Haala

To accommodate the demand for safe transferring of humans and goods in a ship Gangway system, a reliable docking operation is needed. Currently, the docking procedure is still manually achieved, by operator estimating the relative pose between the Gangway and the floating target. For the eventual goal of achieving an automated Gangway system, identifying the pose of the docking target is necessary. In this thesis, a graph-based system is employed to provide robust estimation for the target pose (6-DoF).

A 3D LiDAR scanner and a stereo camera are joined for perception understanding. Both sensors are suited for recovering the pose or movement of docking target with respect to the Gangway, yet both have their limits. The proposed scheme detects the target with the internal algorithm based on sparse 3D point clouds of LiDAR and tracks the pose with two mature odometry algorithms: VISO2 (visual odometry) from Geiger[1] and Iterative Closest Point (ICP), the point–to–plane method (lidar odometry) from the point cloud library [2].

The integrating strategy for the three algorithms is by abstracting observations as a graph and optimizing it. The combined estimation is obtained after local optimization over a small window span to the previous observations. The result of the proposed method is compared to the state-of-the-art SLAM algorithm ORB-SLAM2 [27], which in contrast to our system, is featured with a loop closure.

Groundtruth is often unavailable. To evaluate the result quantitatively, we treat the state-of-the-art visual SLAM system, ORB-SLAM2[3] as our weak groundtruth and estimate the Absolute Trajectory Error (ATE).

Combining the absolute method from U-form detection with either relative method improves the estimated trajectory. The experiments showed the configuration of U-form detection together with VISO2 reached the least absolute trajectory error (ATE) over other combinations and it has the least drift in the crucial docked phase of Gangway operation.

However, due to occlusion in some edge cases, the stereo images alone are not feasible for estimation or failed to estimate. NICP method, on the other hand, raise the availability of the system. The drawback is its problem with low S/N ratio during the docked phase.

Remark: Orb-slam2 in some case failed to provide estimation and drifts as well. The ATE evaluation does not mean the actual accuracy achieved but slightly underestimated. The actual accuracy with the eyeballing evaluation of the estimated position shown on the point cloud dataset shows a 0.05 to 0.1 meter better result than the ATE evaluation.

Both odometry approaches have their advantages and disadvantages at different Gangway operation phases. The critical aspect for gangway operation is to obtain a guaranteed minimum error at any time throughout the operation. Moreover, the system should provide estimation all the time without failure as well. An integrity analysis for different modules in the future is therefore important.

1. Geiger, Andreas, Julius Ziegler, and Christoph Stiller. "Stereoscan: Dense 3d reconstruction in real-time." 2011 IEEE Intelligent Vehicles Symposium (IV). Ieee, 2011.
2. Holz, Dirk, et al. "Registration with the point cloud library: A modular framework for aligning in 3-D." IEEE Robotics & Automation Magazine4 (2015): 110-124.
3. Mur-Artal, Raul, and Juan D. Tardós. "Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras." IEEE Transactions on Robotics5 (2017): 1255-1262.