Geometry based rail track detection and blocking scenario identification using deep learning

Duration of the Thesis: 6 months
Completion: February 2020
Supervisor: Dr.-Ing. Michael Cramer
Prüfer: Prof. Dr.-Ing. Norbert Haala

Storms and snowstorms greatly hinder the regular operation of the trains, which will cause considerable losses to the railway company and significant inconvenience to passengers. Therefore, an essential goal of crisis management after the storm is to clear important corridors between large cities quickly. Hence, it is vital to locate the blocking areas as fast as possible.
At present, hand-operated check for each image is required, which is inefficient and inclined to error. Therefore, the objective of the thesis is to find the blocking scenarios on the tracks automatically.

Before the processing part, the Google Map API was firstly utilised to delete the images, which are near the train station area, from the dataset.
Then, a series of image processing methods were implemented to increase the image quality and prepare for the subsequent geometry-based line detection method.

The aim of the geometry-based line detection method was to detect the rail tracks in the image to replace the manual labelling partially.
After labelling, a two-step segmentation method was used to improve the processing speed for the specific dataset used in the thesis: firstly, the reduced-size images are taken advantage for the small-scale railway area segmentation; secondly, the coordinates of the bounding boxes in the small pictures were reversed to the original size. Large-scale instance segmentation could be performed using only rectangular bounding box areas in the original images. Mask-RCNN was implemented here for both small-scale instance segmentation and large-scale instance segmentation.

At last, if the probability of the instance in the photo being identified as a track was lower than usual, this picture would be marked as a problematic picture containing possible blocking scenes and must be checked manually(see figure 2).

Figure1 demonstrates an overview of the project process flow.

This thesis provides a solution to find the abnormal region of the railway through image processing and deep learning to reduce the burden of manual operations.

Another advantage of this solution is that it can be regarded as a semi-real-time image detection method. The training time of the entire dataset does not exceed two hours. Furthermore, if there is a pre-trained model, it takes only about one second to process an image.

There are some possible ways to improve this solution in the future.

Firstly, only the location data where the pictures were taken and the content of the images itself was used in the processing. Internal and external orientation parameters of the photos can be taken into consideration, which might be helpful to establish the connection between the images and reduce the stress of image processing.

Secondly, if Deutsche Bahn provides the DB coordinates, the locations of the railways, the crossings and the tunnels can be back-projected to the image. It will greatly reduce the preparation before the training step.

Last but not least, there were no actual blocking scenes, so the breakpoints were artificially added to simulate the blocking situation. Therefore, it requires more tests, especially after the storm.