Semantic segmentation for South African Railway detection

Abstract

Railway detection in images is a catalyst required to advance the railway industry towards autonomous trains; to enable automatic remote inspections of the railway from video footage; and to create an advance warning system for railway obstacles. Despite the benefits of railway detection, it is relatively unexplored territory and has only recently garnered attention.

This research aims to apply semantic segmentation models to South African railway images to achieve railway detection. As a prerequisite to the application, a dataset of 1000 South African railway images were collected and labelled.

Current railway detection methods are documented in the following papers: RailSem19, Railway Intrusion Detection and RailNet. These were analysed to determine the most suitable network for implementation. RailNet was found to achieve the highest mean IoU score of 89.8% amongst the networks reviewed and consequently selected to emulate.

The collected dataset was used to train the RailNet network as well as the SegNet and UNet network architectures in a configuration of 600 training images; 200 validation images and 200 test images. The SegNet, UNet and RailNet networks scored a mean IoU score of 90.32%, 92.93% and 92.99% respectively.

A total of 600 images from a publicly available dataset (RailSem19) were selected to supplement the South African training dataset to determine the effect of the foreign data on the performance of the networks. The additional data had an overall positive effect on the performance of the networks, increasing the baseline mean IoU values of SegNet, UNet and RailNet to 96.26%, 96.36% and 96.81% respectively.

The test set images were divided into seven (7) categories to determine the relationship between the networks performance and the terrain, luminosity or number of rails in the image. SegNet and UNet achieved the highest mean IoU score for the ‘miscellaneous’ and ‘rocky’ category respectively, while RailNet dominated the remaining five (5) categories: bright, dark, single rail, double rail and vegetation.

The application, Labelme, was used to create pixel-wise labels, segmenting the railways from the background in the South African images.

All networks in this research were implemented in Python, using the Keras Library with a Tensorflow backend and trained using Google Colab, making use of the GPU capabilities available

The contributions of this research are: a labelled dataset of 1000 South African images for semantic segmentation of railways; determining the performance of RailNet (the current best performing rail detection network) on the South African dataset; comparing this performance with SegNet and UNet network architectures to establish the benchmark performance; determining the effect on the performance resulting from supplementing the South African dataset with foreign data; and investigating the relationship between the networks performance in relation to the the terrain, luminosity or number of rails in the image.