Professional Documents
Culture Documents
of the 2017 IEEE International Conference on Signal and Image Processing Applications (IEEE ICSIPA 2017), Malaysia,
September 12-14, 2017
Abstract—Visual inspection of rail fasteners is crucial to systems can be used with cameras installed under automatic
rail safety. However, the traditional method in which railway rail inspection vehicles.
staffs manually inspect the conditions of fasteners is time-
consuming and prone to human error. In this paper, we present
A top-down approach is used in our method to classify
a method to automatically detect missing rail fasteners from the status of the fasteners in each image. The method can
top-view images. Using a top-down approach, coarse bounding be roughly divided into two steps: fastener region estimation
boxes of potential fastener areas are first located from the and missing fastener detection. In the first step, coarse
track and the tie regions with an edge density map and the regions of possible fastener locations are estimated from the
RANSAC algorithm. Preprocessed with the guided filter, the
region within the bounding boxes are then scanned to detect
track and the tie using edge density map. The regions are
rail fasteners using PHOG features and -SVR with RBF located at each side of the intersection between the lines
kernel. The boxes, in which no fasteners are found, are reported that represent the track and tie. The located regions are then
as missing fasteners. The proposed method was tested and has scanned in the next step to determine whether each region
shown a degree of robustness in scenes from complex real- contains a fastener. This is done to detect existing fasteners
world environments with the 100% probability of detection and
3.47% probability of false alarm for missing fastener detection.
using PHOG features and -SVR. Finally, if no fasteners are
The results also indicate that the use of guided filter, RBF detected, a missing fastener report is filed.
kernel and the image pyramid technique for feature extraction The remaining part of this paper is organized into 6
significantly improves the performance of the classifier. sections. In section II, a survey of current methods in
missing rail fastener detection is presented. Next, the image
I. I NTRODUCTION acquisition process and the dataset used in this study are
Rail fastener is a crucial component in rail systems that described in section III. Details of fastener region estimation
fixes rail tracks and ties together. For concrete ties, two types and missing fastener detection are presented in section IV
of rail fasteners: hexagonal-headed bolts, and hook-shaped and section V respectively. In section VI, we show the
fasteners or rail clips, are commonly used [1]. Defects in experimental results from our proposed method as it was
and absence of the component, as shown in Fig. 1b, can tested in real-world environments. We conclude our findings
be the cause of serious injuries such as train derailment. and discuss the future work in section VII.
Thus, constant monitoring of rail fasteners conditions is
essential to ensure safety. However, rail fastener inspection II. R ELATED W ORK
is traditionally done in term of visual inspection by human
i.e. railway staffs walk along the railway to inspect whether Since the early 2000s, a number of vision-based methods
there are any defects in fasteners and other rail components. for rail fastener inspection have been proposed. Stella et al.
Such method of inspection consumes a huge amount of time [2] presented a wavelet transform-based method to recognize
and is prone to human errors. As a result, automatic rail hexagonal-headed rail-fastening bolts and rail clips using a
inspection vehicles are becoming available. 3-layer neural classifier. Singh et al. [3] used edge density
In this paper, we aim to develop an algorithm to detect for detecting missing fasteners from videos. Marino et al. [4]
missing rail fasteners from top-view images of rail ties. Such proposed the VISyR system for detecting hexagonal bolts
in real-time using multilayer perceptron neural classifiers
(MLPMCs). In [5], Xia et al. presented an approach to
detect broken rail fasteners with a method based on Haar-like
features and the Adaboost algorithm by dividing the fastener
region into several parts. Yang et al. [6] employed direction
fields as the feature to detect missing rail clips from images.
In [1], Feng et al. utilized structural topic model to model the
fastener a method for the inspection of rail fasteners. Harris-
Stephen and Shi-Tomasi feature detectors were also used by
Khan et al. [7] to detect missing rail clips in image. In [8],
(a) intact fasteners (b) missing (left) and intact fasten-
ers (right) Gibert et al. proposed a method to assess and classify rail
fasteners using Histogram of Gradient (HOGs) and a group
Fig. 1: Examples of intact and missing rail fasteners. of Support Vector Machines (SVMs) with linear kernel.
420
Proc. of the 2017 IEEE International Conference on Signal and Image Processing Applications (IEEE ICSIPA 2017), Malaysia,
September 12-14, 2017
421
Proc. of the 2017 IEEE International Conference on Signal and Image Processing Applications (IEEE ICSIPA 2017), Malaysia,
September 12-14, 2017
422
Proc. of the 2017 IEEE International Conference on Signal and Image Processing Applications (IEEE ICSIPA 2017), Malaysia,
September 12-14, 2017
423
Proc. of the 2017 IEEE International Conference on Signal and Image Processing Applications (IEEE ICSIPA 2017), Malaysia,
September 12-14, 2017
VII. C ONCLUSION Fig. 11: Example of the missing fastener detection results
In this paper, we have presented a method for automat- from the proposed classifier. The colored rectangles indicate
ically detecting missing rail fasteners from images. First, the region with the highest score in that image with the
a coarse fastener region estimation method based on edge score denoted on top. The red and green rectangles signify
density map and RANSAC is performed in order to lessen positive (missing fasteners) and negative (intact fasteners)
the search space for the fine detection of fasteners. Then, the classifications respectively. Orange rectangles denote the
missing fastener detection method based on PHOG features ground truth of the fastener bounding boxes. True negatives,
and -SVR are applied to the region obtained from the region true positives, false negatives, and false positives are shown
estimation step. Guided filter is also used to smooth the in (a) to (e), (f) to (h), (i) to (k), and (l) respectively.
texture of rail fasteners before further processing. Our testing
has shown that the proposed classifier is robust to a degree of
[7] R. A. Khan, S. Islam, and R. Biswas, “Automatic detection of
occlusions and variance in texture of the fastener observed defective rail anchors,” in 17th International IEEE Conference on
in complex real-world environment. Intelligent Transportation Systems (ITSC), 2014, pp. 1583–1588.
Future work includes improving the method to better [8] X. Gibert, V. M. Patel, and R. Chellappa, “Robust Fastener Detection
for Autonomous Visual Railway Track Inspection,” in 2015 IEEE
handle scenes with major occlusions and texture anomalies. Winter Conference on Applications of Computer Vision, 2015, pp.
Performance optimization is also needed to make the algo- 694–701.
rithm able to work in real time. [9] X. Gibert, V. M. Patel, and R. Chellappa, “Deep Multitask Learning
for Railway Track Inspection,” IEEE Transactions on Intelligent
R EFERENCES Transportation Systems, vol. 18, no. 1, pp. 153–164, 2017.
[10] N. Dalal and B. Triggs, “Histograms of oriented gradients for human
[1] H. Feng, Z. Jiang, F. Xie, P. Yang, J. Shi, and L. Chen, “Automatic detection,” in 2005 IEEE Computer Society Conference on Computer
Fastener Classification and Defect Detection in Vision-Based Railway Vision and Pattern Recognition (CVPR’05), vol. 1, Jun. 2005, pp.
Inspection Systems,” IEEE Transactions on Instrumentation and 886–893 vol. 1.
Measurement, vol. 63, no. 4, pp. 877–888, Apr. 2014. [11] A. Bosch, A. Zisserman, and X. Munoz, “Representing Shape
[2] E. Stella, P. Mazzeo, M. Nitti, G. Cicirelli, A. Distante, and with a Spatial Pyramid Kernel,” in Proceedings of the 6th ACM
T. D’Orazio, “Visual recognition of missing fastening elements for International Conference on Image and Video Retrieval, ser. CIVR
railroad maintenance,” in Proceedings. The IEEE 5th International ’07. New York, NY, USA: ACM, 2007, pp. 401–408. [Online].
Conference on Intelligent Transportation Systems, 2002, pp. 94–99. Available: http://doi.acm.org/10.1145/1282280.1282340
[3] M. Singh, S. Singh, J. Jaiswal, and J. Hempshall, “Autonomous Rail [12] K. He, J. Sun, and X. Tang, “Guided image filtering,” IEEE Trans-
Track Inspection using Vision Based System,” in 2006 IEEE In- actions on Pattern Analysis and Machine Intelligence, vol. 35, no. 6,
ternational Conference on Computational Intelligence for Homeland pp. 1397–1409, June 2013.
Security and Personal Safety, 2006, pp. 56–59. [13] N. Otsu, “A threshold selection method from gray-level histograms,”
[4] F. Marino, A. Distante, P. L. Mazzeo, and E. Stella, “A real-time visual IEEE Transactions on Systems, Man, and Cybernetics, vol. 9, no. 1,
inspection system for railway maintenance: Automatic hexagonal- pp. 62–66, Jan 1979.
headed bolts detection,” IEEE Transactions on Systems, Man, and [14] M. A. Fischler and R. C. Bolles, “Random sample consensus: A
Cybernetics, Part C (Applications and Reviews), vol. 37, no. 3, pp. paradigm for model fitting with applications to image analysis and
418–428, May 2007. automated cartography,” Commun. ACM, vol. 24, no. 6, pp. 381–395,
[5] Y. Xia, F. Xie, and Z. Jiang, “Broken Railway Fastener Detection Jun. 1981.
Based on Adaboost Algorithm,” in 2010 International Conference on [15] H. Drucker, C. J. C. Burges, L. Kaufman, A. Smola, and V. Vapnik,
Optoelectronics and Image Processing, vol. 1, 2010, pp. 313–316. “Support vector regression machines,” in Proceedings of the 9th
[6] J. Yang, W. Tao, M. Liu, Y. Zhang, H. Zhang, and H. Zhao, “An International Conference on Neural Information Processing Systems,
Efficient Direction Field-Based Method for the Detection of Fasteners ser. NIPS’96. Cambridge, MA, USA: MIT Press, 1996, pp. 155–161.
on High-Speed Railways,” Sensors, vol. 11, no. 8, pp. 7364–7381, Jul.
2011.
424