Masked Based 6D Vision Stixel Tracking

Master Thesis at ifp - Vanessa Tietz

Vanessa Tietz

Masked Based 6D Vision Stixel Tracking

Duration of the Thesis: 6 months
Completion: June 2020
Supervisor: Dr. Dominik Müller (Robert Bosch GmbH)
Examiner: Prof. Dr.-Ing. Norbert Haala


 

Motivation and Aim of the Thesis

In the field of autonomous driving the environmental recognition is a challenging task. It can be conducted using different sensors like a stereo camera. A drawback of using a stereo camera is the huge amount of data which have to be processed. Every pixel and the corresponding depth information leads to millions of data points. To cope with that the Stixel World was introduced by Badino [1]. In the Stixel World pixels are grouped to rectangular super pixels with a fixed width and the height according to the objects as it can be seen in figure 1.

Figure 1: Multi-Layer Stixels

Every Stixel contains information about its position and velocity. This approach has been continuously evolved but still has some drawbacks due to a lot of model assumptions and complex calculation algorithms. Additionally, the image processing is conducted global, which means that the whole image has to be processed.
With a new available input from a Single-Shot-Detection neural network in the sense of bounding boxes with their semantics and so called masks it is possible to do the image processing locally in the sense of only processing the bounding boxes. The bounding boxes and the corresponding masks can be seen in figure 2.

Figure 2: Bounding Boxes with Masks

The masks provide more precise information whether a pixel in the bounding box belongs to the corresponding object which leads to a better shape approximation of the object. The bounding boxes are divided into rectangles.
This thesis addresses the calculation of that novel Stixels based on the new input to improve the shape approximation of objects based on the more precise disparity information as well as the determination of a velocity of each Stixel.

Research Tasks

Newly generated Stixels with corresponding velocity measurement- and tracking-algorithms
are implemented based on the new input within this thesis. The method the Stixels are generated was changed as well as the measurement and tracking algorithm for determining the motion. This thesis focused on the following tasks:

  • Implement new Stixel calculation algorithms to approximate the shape of obstacles based on the disparity values. The disparity values are provided by a Semi-Global-Matching algorithm. One shape approximation can be seen in figure 3 where the blue marks are the disparity values of a Stixel. Every Stixel receives one disparity value, in case of that spline algorithm the largest disparity value of the spline.
Figure 3: Shape approximation with spline
  • Implement different measurement and tracking algorithms of each Stixel to determine a velocity for each Stixel. One of these algorithms is visualized in figure 5 where the feature points provided from a KLT-Tracker (Kanade-Lucas-Tomasi) were used to determine the velocity based on the difference quotient. Subsequently tracking the velocity by using a moving average exponential filter.
Figure 4: Measurement and Tracking of Stixels
  • Compare the new approach with the state-of-the-art approach using different video ground truth sequences. The following figure shows a ground truth bounding box with its velocity in blue, the state-of-the-art Stixels with their velocity in red and the new (Zorro) Stixels with their velocity in green.
Figure 5: Benchmarking with ground truth object in blue

Conclusion

Using the new input data, it was possible to improve the position of the Stixels with all implemented algorithms in comparison with the state-of-the-art Stixels. Regarding the velocity estimation, it was possible to improve the quality with one of the implemented algorithms in two of three ground truth video sequences. Possible further research tasks could be the parallelization of the program code and a runtime determination in comparison with the state-of-the-art approach.

Ansprechpartner

Dieses Bild zeigt Norbert Haala

Norbert Haala

apl. Prof. Dr.-Ing.

Stellvertretender Institutsleiter

Zum Seitenanfang