Calibration of automotive fish-eye cameras with misaligned camera components

Dauer der Arbeit: 6 Monate
Abgabe: März 2014
Betreuer: Dr. Michael Cramer, Dr. Jagmal Singh (Magna Electronics, Aschaffenburg)
Prüfer: Prof. Dr.-Ing. Norbert Haala

In automotive industry the 3D perception of the vehicle's surroundings is becoming increasingly important both for car manufacturers and for customers, as it provides crucial information for the sake of passenger's safety, driver's assistance and autonomous driving systems. As more and more surrounding information and a wider range of functionality are demanded, the automotive vision systems by modern standard have to cover a 360° view, meanwhile offering some image processing features such as feature tracking and image stitching in real time.

The compact fish-eye cameras have been brought to attention because their manufacturing cost is considerably decreased in recent years, and they are small enough to be easily mounted on the vehicle without noticeable appearance. The field of view of a fish-eye camera is larger than a normal central projection system, therefore the full surrounding view can be covered by fewer fish-eye cameras.

The fish-eye camera however introduces very strong curvilinear distorting effect, and the data extracted from the image do not match the geometric information of the real scene. Also the compact fish-eye cameras suffer more in accuracy and image quality from the manufacturing errors because of their size and cost-effective production.

One of the errors that stand out is the misalignment of camera components. For a high-end camera the misalignment is normally ignored since it is relatively small. For the camera of automotive standard this type of error must be considered, otherwise the camera cannot be successfully calibrated.

Experiments

Other than the misalignment there could be other errors of smaller significance but with considerable amount of cumulative effect. In order to fully calibrate the cameras, a model that not only covers all of these errors must be devised. The calibration model must also adapt to the manufacturing process of the cameras, and the embedded on-board system should be able to include the model without performance issue. And the end of calibration error must be maintained under sub-pixel level.

In this thesis, a model consisting of a spherical projection model and a bivariate polynomial model is proposed for the calibration of the automotive fish-eye camera with the resolution 1280 × 800. In the calibration test with a target field designed for mass-production the proposed model has been proved to be valid.

Figure 2 shows the residual errors of target points after calibration. The model has been used to calibrate the “distortion” ̶ transformation of undistorted image to distorted image, the “undistortion” ̶ transformation of distorted image to undistorted image, and both at the same time. Most of the points have residual errors under 0.4 pixel × 0.4 pixel, and the sub-pixel level of accuracy is achieved.

In order to find out if the calibration result is also valid for non-target points, a successive “undistortion” and “distortion” process is performed. The result is shown in Figure 3. The reprojection errors for most points are under 0.5 pixel, and thus the sub-pixel accuracy still holds true for most of the image area.

The model has been tested for a series of cameras from the same batch of manufacturing, which were calibrated before by a model that does not consider misalignment. The improvement of the proposed model can be seen in Figure 4. The previously used model was not able to keep the reprojection error under 1 pixel for more than half of the tested cameras, while the proposed model can maintain the error universally under 0.2 pixel. That means the misalignment has been well corrected by the new model.

In the meantime the proposed model has been tested against a high-end fish-eye camera system composed of the Nikon D800E camera and the AF Fisheye-NIKKOR 16mm f/2.8D lens. The result presented in Figure 5 is compared to that of the automotive fish-eye camera in Figure 6. Both cameras can be calibrated in general with sub-pixel accuracy for most target points. The calibration for the Nikon system exhibits better result than the automotive camera. The automotive camera has produced incorrect reprojection at the lower left corner. This is partially because the target points at the corner is severely distorted and the light environment is unfriendly to the automotive camera. Other than that, the automotive camera contains larger magnitude of errors than the Nikon system.

The main objective of this thesis is to find out a camera model that is able to correct the misalignment error of a series of automotive fish-eye camera used in the automotive vision system. A model based on polynomial and a spherical projection was proposed, and it has been validated for being able to correct the misalignment with an average residual error of 0.2 pixel. In the meantime it is able to satisfy the accuracy and functionality requirements of the vision system and keep up to the industrial standard for the calibration process.

In comparison with the model that does not consider misalignment, the newly proposed model has more parameters and requires more calibration time, but it is worthwhile due to the improvement and the quality test for accuracy can be guaranteed. The model can also be used for calibration of other fish-eye cameras such as the Nikon system, and the calibration accuracy for the high-end system is also improved.