Road sign detection

Andreas Mgelmose

February 14, 2012

Introduction
Detection and recognition
Road sign detection Andreas Mgelmose
2

Introduction
Driver support Sign examples Method categorization

Trac Sign Recognition (TSR) has several use cases: Driver assistance Autonomous cars Sign inventory and maintenance The task of TSR has two logic blocks: Sign detection Sign recognition Usually the systems doing this are split into these two blocks as well. Ill mostly talk about detection.

Methods
Segmentation Feature extraction Detection

What works? Open problems Detection examples Synthetic training

19

Driver support
Cant people just read the signs?
Road sign detection Andreas Mgelmose Introduction
3 Driver support Sign examples Method categorization

Apparently not:
Signicant results from [4] regarding attention to various sign types:
Fixated Recalled Not recalled 100 60 8 0 0 0 54 21 Not xated Recalled Not recalled 0 7 0 0 0 33 38 79

Methods
Segmentation Feature extraction Detection

What works? Open problems Detection examples Synthetic training

Target Speed limit 80 km sign Game Crossing sign Pedestrian crossing ahead Pedestrian crossing sign

19

Driver support
Adaptive detection
Road sign detection Andreas Mgelmose Introduction
4 Driver support Sign examples Method categorization

Regular detection for autonomous cars:

Methods
Segmentation Feature extraction Detection

What works? Open problems

Detection in a driver-in-the-loop system:

Detection examples Synthetic training

19

Sign examples
Road sign detection

Dierent shapes and colors:

5

Andreas Mgelmose Introduction

Driver support Sign examples Method categorization

Methods
Segmentation Feature extraction Detection

What works? Open problems

More colors, more shapes, lots of text:

Detection examples Synthetic training

... seriously, USA?

19

Detection methods
The old categorization
Road sign detection Andreas Mgelmose Introduction

Traditionally, two general detection approaces: Color-based detection

Easy to implement Easy (if laborious) to train Vulnerable to changes in lighting conditions. Does not work with uncolored signs.

Driver support Sign examples 6 Method categorization

Methods
Segmentation Feature extraction Detection

What works? Open problems Detection examples Synthetic training

Shape-based detection
Lighting invariant (as long as there is light) Can detect signs of various colors. More complicated to implement. Slow(er).

19

Detection methods
A new categorization scheme
Road sign detection Andreas Mgelmose Introduction
Driver support Sign examples

Most modern systems use a combined approach.

Method categorization

Methods
Segmentation Feature extraction Detection

We propose a new categorization scheme: Segmentation Feature extration Detection

What works? Open problems Detection examples Synthetic training

19

Segmentation
Road sign detection

Thresholding in various color spaces:

Andreas Mgelmose Introduction

Driver support Sign examples Method categorization

Methods
8 Segmentation Feature extraction Detection

What works? Open problems Detection examples

Other color based methods such as quad-tree color selection [3]. Saliency and biologically based. Edges (see next slides). No segmentation. See more in Gmez-Moreno et. al., 2010 [1].

Synthetic training

19

Feature extraction
Road sign detection Andreas Mgelmose Introduction

Edges:

Driver support Sign examples Method categorization

Methods
Segmentation 9 Feature extraction Detection

What works? Open problems Detection examples Synthetic training

Histogram of Oriented Gradients (HOG). Haar wavelet-like features. Others, such a Distance to Bounding box (DtB).

19

Detection
Road sign detection

The detection is of course tightly coupled with the choice of features. Edges: Hough transform, Extended Fast Radial Symmetry

Andreas Mgelmose Introduction

Driver support Sign examples Method categorization

Methods
Segmentation Feature extraction 10 Detection

What works? Open problems Detection examples Synthetic training

HOG: Support Vector Machines Haar wavelets: Cascaded classier Other (rarely used) options are neural networks and genetic algorithms.
19

What works?
Nobody really knows
Road sign detection Andreas Mgelmose Introduction

Standardized data sets only emerged recently: Comparison between results is nearly impossible. Luckily, a few have shown up: German Trac Sign Recognition Benchmark [5] - 41000 images, mostly for classication. Created as foundation for a competition. Swedish Trac Signs Dataset [2] - 20000 images (4000 labeled) and video tracks available. UCSD Trac Sign Database - it is going to be awesome :-) The trend seems to go towards learned detection algorithms.
11

Driver support Sign examples Method categorization

Methods
Segmentation Feature extraction Detection

What works? Open problems Detection examples Synthetic training

19

Open problems
Road sign detection Andreas Mgelmose Introduction
Driver support Sign examples Method categorization

Methods

US sign detection and classication. Does recognized signs belong to the current road? How do we quickly adapt a car to new countries with new signs? Proper integration of TSR into driver support systems.
12

Segmentation Feature extraction Detection

What works? Open problems Detection examples Synthetic training

19

My project
Road sign detection Andreas Mgelmose Introduction

Survey of existing work

Driver support Sign examples Method categorization

Methods

Detectors

Segmentation

AdaBoost/ViolaJones

SVM

Fast radial symmetry

13

Feature extraction Detection

What works? Open problems Detection examples

Training data

Synthetic data

Real data

Synthetic training

Detector recommendation

19

Detection of diamond shapes

Road sign detection Andreas Mgelmose Introduction
Driver support Sign examples Method categorization

Methods
Segmentation Feature extraction Detection

What works? Open problems

14

Detection examples Synthetic training

19

Detection of octagons
Road sign detection Andreas Mgelmose Introduction
Driver support Sign examples Method categorization

Methods
Segmentation Feature extraction Detection

What works? Open problems

15

Detection examples Synthetic training

19

Detection of speed limit signs

Road sign detection Andreas Mgelmose Introduction
Driver support Sign examples Method categorization

Methods
Segmentation Feature extraction Detection

What works? Open problems

16

Detection examples Synthetic training

19

Synthetic training
Road sign detection Andreas Mgelmose Introduction
Driver support Sign examples Method categorization

Methods
Segmentation Feature extraction Detection

Obtaining thousands of images of signs is time consuming and incredibly boring. Can we automate the process? Well, perhaps. Let me show you.
17

What works? Open problems Detection examples Synthetic training

19

Bibliography I
Road sign detection Andreas Mgelmose

H. Gomez-Moreno, S. Maldonado-Bascon, P. Gil-Jimenez, and S. Lafuente-Arroyo. Goal Evaluation of Segmentation Algorithms for Trac Sign Recognition. Intelligent Transportation Systems, IEEE Transactions on, 11(4):917930, dec. 2010. F. Larsson and M. Felsberg. Using fourier descriptors and spatial models for trac sign recognition. Image Analysis, pages 238249, 2011. Andrzej Ruta, Fatih Porikli, Shintaro Watanabe, and Yongmin Li. In-vehicle camera trac sign detection and recognition. Machine Vision and Applications, 22:359375, 2011.
19

Introduction
Driver support Sign examples Method categorization

Methods
Segmentation Feature extraction Detection

What works? Open problems Detection examples

18

Synthetic training

Bibliography II
Road sign detection Andreas Mgelmose Introduction
Driver support Sign examples

David Shinar. Trac safety and human behaviour. Emerald Group Publishing, 2007. J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel. The German Trac Sign Recognition Benchmark: A multi-class classication competition. In Neural Networks (IJCNN), The 2011 International Joint Conference on, pages 14531460. IEEE, 2011.

Method categorization

Methods
Segmentation Feature extraction Detection

What works? Open problems Detection examples

19

Synthetic training

19