Development of an Algorithm for Ghost Detection in the Context of Stray Light Test

Duration of the Thesis: 6 months
Completion: February, 2016
Supervisor & Examiner: Prof. Dr.-Ing. Norbert Haala
Tutor: Dr. Uwe Apel (Robert Bosch GmbH)

HDR CMOS cameras used in video based driver assistance systems are designed to acquire the outperforming wide luminance range between the darkest and the brightest of night sceneries. However, its performance is especially critical in case of being exposed in glaring situation with uncontrollable illumination, for example, oncoming car’s headlights and reflection at traffic signs. Under these circumstances unwanted artifacts can be acquired in the images because of stray light caused by mechanical defects, impurities or imperfect image sensor or lens which are inevitable in actual production. At present, there is no automated method based on image processing tools to identify the ghost artifacts in the HDR images obtained from the camera systems. In this thesis project, an algorithm used to detect the ghost artifacts has been developed and evaluated for the end-of-line test of the cameras.

Ghost Categories

Ghost artifacts are changing with the variation of the position of light source spot. The ghost artifacts can be distinguished according to their shape, brightness, size and completeness. The following categories were classified subjectively by observation of human eyes. The processing and analysis were based on these categories.

Based on the common features of ghost artifacts in the whole test images and the requirements of this thesis, the algorithm was designed in three main sections using Matlab. First, automatically detect whether visible ghost artifact exists in the image. If it exists, then segment the ghost region from the whole image as a sub-image used in next step. Secondly, extract the higher intensity pixels of the ghost artifact in a shape of a part of ring, which is named “Ghost Ring” in this thesis. At last, calculate the mean values of “Ghost Ring” as ghost feature, which will be integrated in the final stray light features set.

Ghost Region Detection

In the preprocessing part, build a round mask to cover the spot in the test image first. Then, several connected components are acquired after applying noise reducing, image smoothing, edge detection and morphology processing on the image with mask in sequence. Next, the ghost’s type is judged according to the distance between the spot and centroids of those connected components, and the number of the connected components that larger than the defined area. After the judgment, the region with normal type ghost is segmented as sub-image. Here, an image within a typical ghost artifact in middle intensity is used as an example to show the image processing steps.

Firstly, extract some points inside the high intensity part of ghost to roughly represent the shape of ghost. And then build a ring that contains the significant part of ghost based on circle fitting using these points. Because the significant part of ghost just takes a quarter of the entire ring approximately, this region has to be extracted to supply a basis for ghost feature computation. In order to detect the significant part of ghost, a special point that in the middle of the significant part is designed to extract first. With the extension from the point towards both sides along the ring, the suitable quarter of ring is formed. This quarter part of ring is named “Ghost Ring” in this thesis.

The main functionality of this module is carrying out calculation of the features of ghost artifact in order to provide a data support for final analysis. It is an efficient method to understand the ghost stray light performance in the filters sub-sampling mode based on the special property of the “CRBC” filter pattern of imager. First, separate the pixels of the ghost sub-image into four single filter patterns according to their position relationship in each block. Next, calculate the gray value histograms of the “Ghost Ring” with respect to the four separated filter patterns, these will explain the visual discrepancies in detailed gray value data and will be helpful for an analysis in the opinion of physical.

Conclusion

Within the scope of a feasibility study, this thesis proposes a new algorithm for automatically detecting ghost artifacts and extracting ghost feature in the context of stray light test for the video camera MPC2 from Bosch. Results from 118 images from 4 test cameras show that, this study has achieved the priority goals: detecting ghosts, judging ghosts and extracting ghost’s feature, which equal with replacement of the subjective ghost detection by eyes with an objective procedure using software.

References

1. Hoefflinger, B. "High-Dynamic-Range (HDR) Vision: Microelectronics, Image Processing." Computer Graphics (Springer Series in Advanced Microelectronics), Springer-Verlag New York, Inc., Secaucus, NJ, 2007.
2. Gonzalez, Rafael C. Digital Image Processing. Pearson Education India, 2009.
3. Raizner, Carina. Objective and Automated Stray Light Inspection of High-Dynamic-Range Cameras. Shaker, 2012.