Segmenting the License Plate Region Using a Color Model 401

X

Segmenting the License PIate Region
Using a CoIor ModeI

Kaushik Del and Kang-Hyun }o
Unitcrsi|q cf U|san
Scu|n Kcrca

1. Introduction
Hunans can perforn usuaI largel recognilion vilhoul loo nuch efforl. Hovever, ly
conpuler lhe lask of recognizing specific oljecl in an inage, one of lhe nosl difficuIl lopics
in lhe fieId of conpuler vision or digilaI inage processing. VehicIe Iicense pIale deleclion
(VLID) lask is quile chaIIenging fron vehicIe inages due lo lhe nuIli-slyIe pIale fornals,
viev poinl changes and lhe nonuniforn ouldoor iIIuninalion condilions during inage
acquisilion (AnagnoslopouIos el aI., 2OO8) and (}iao el aI., 2OO9). In addilion, VLID syslen
shouId operale fasl enough (reaI line) lo salisfy lhe needs of inleIIigenl lransporlalion
syslens (ITSs) and nol lo niss a singIe inleresl oljecl fron lhe vehicIe inage. VLID is aIso
very inleresling in finding Iicense pIale area fron vehicIe inage. The VLID is videIy used
for delecling speeding cars, securily conlroI in reslricled areas, in unallended parking zones,
for lraffic Iav enforcenenl and eIeclronic loII coIIeclion, elc. Wilh lhe rapid deveIopnenl of
highvay and lhe vide use of vehicIes, peopIe have slarled lo pay nore and nore allenlion
lo lhe advanced, efficienl, and accurale ITSs. RecenlIy, lhe necessily of vehicIe Iicense pIale
recognilion (VLIR) has increased significanlIy. The Iicense pIale deleclion is a cruciaI and
indispensalIe conponenl of VLIR syslen. One of lhe najor prolIens in LI deleclion is
delernining LI syslens. This syslen nusl guaranlee rolusl deleclion under various
vealher and Iighling condilions, independenl of orienlalion and scaIe of lhe pIale.
In lhe recenl years deveIopnenls deaIing vilh sinpIe inages have leen achieved vilh
acceplalIe resuIls. Hovever, recenl researches have leen addressed lo processing conpIex
inages vilh unconslrained condilions (Malas, 2OO5). The proposed Iicense pIale deleclion
franevork deaIs vilh such vehicIe inages.
In lhis proposed VLID nelhod, consisls of lvo nain slages. IniliaIIy, HSI coIor nodeI is
adopled for delecling candidale regions. According lo differenl coIored LI, lhese candidale
regions nay incIude LI regions, geonelricaI properlies of LI are lhen used for
cIassificalion. The proposed nelhod is alIe lo deaI vilh candidale regions under
independenl orienlalion and scaIe of lhe pIale. More lhan one Iicense pIale can le delecled
in lhe sane inage. IinaIIy, lhe deconposilion of candidale regions conlain predelernined
LI aIphanuneric characlers ly using posilion in lhe hislogran lo verify and delecl vehicIe
Iicense pIale region.
22
Pattern Recognition 402

The focus of lhis chapler is on lhe consoIidalion of a nev nelhod lo seIecl aulonalicaIIy
slalislicaI lhreshoId vaIue in HSI coIor nodeI for delecling candidale regions. CeneraIIy, as a
connon vay of coIor-lased VLID syslen, lhreshoId vaIue is defined ly predelernined
coefficienls or ly user. Il provides slalIe resuIl, lul in poor Iighling condilion il is loo
sensilive. Whereas in our experinenls ve caIcuIale lhreshoId vaIue in a slalislicaI vay, 2O
of sanpIe dala (onIy green, yeIIov and vhile LI areas) are randonIy seIecled for lraining.
Afler lraining fron lhose sanpIe dala, lhe nean and slandard devialion vaIues of hue are
conpuled for deleclion of green and yeIIov LI pixeIs. Delecling vhile Iicense pIale pixeIs,
lhe nean and slandard devialion vaIues of saluralion and inlensily are conpuled lo delecl
green, yeIIov and vhile LI fron vehicIe inages.
In addilion, lhe proposed nelhod is alIe lo deaI vilh pIales (candidale regions) under
independenl orienlalion and scaIe of lhe pIale. More lhan one Iicense pIale can le delecled
in lhe sane inage. Iurlhernore, candidale regions nay incIude LI regions, geonelricaI
properlies of LI are lhen used for cIassificalion. IinaIIy, lhe deconposing of candidale
region vhich conlains predelernined LI aIphanuneric characler, ly using posilion in lhe
hislogran lo verify and delecl vehicIe Iicense pIale region is perforned.

2. ReIevant work
This seclion provides a descriplive sunnary of sone nelhods lhal have leen inpIenenled
and lesled for VLID. As far as deleclion of lhe pIale region is concerned, researchers have
found nany nelhods of Iocaling Iicense pIale. Ior exanpIe, survey paper (AnagnoslopouIos
el aI., 2OO8), offers lo researchers a Iink lo a pulIic inage dalalase lo define a connon
reference poinl for VLIR aIgorilhnic assessnenl. In addilion, lhis survey paper discusses
aloul currenl lrends and anlicipaled research in VLIR syslen. In (AnagnoslopouIos el aI.,
2OO6), a nelhod lased on inage segnenlalion lechnique naned as sIiding vindovs (SW)
has aIso leen proposed for delecling candidale region (LI region). The nain lhoughl of
inage segnenlalion lechnique in LI can le vieved as irreguIarilies in lhe lexlure of lhe
inage and lherefore alrupl changes in lhe IocaI characlerislics of lhe inage, nanifesl
prolalIy lhe presence of an LI. A convenlionaI slalislicaI cIassifier, lased on lhe k nearesl
neighlor ruIe, is used lo cIassify every pixeI of a lesl inage lo ollain a pixeI nap vhere
group of posilive sanpIes prolalIy indicales lhe Iocalion of a Iicense pIale. In lhis syslen,
line-consuning lexlure anaIysis is presenled in (Cano & Ierez-Corles, 2OO3), vhere a
conlinalion of a kd-lree dala slruclure and an approxinale nearesl neighlor vas
espoused. The conpulalionaI resource denand of lhis segnenlalion lechnique is lhe nain
dravlack, laking an average of 34 seconds lo process of singIe inage. In (Chacon &
Zinnernan, 2OO3), lhe puIse-coupIed neuraI nelvork (ICNN) is proposed lo generale
candidale regions lhal nay conlain a Iicense pIale. If lhe Iicense pIale is nol Iocaled in lhe sel
of candidale regions, lhe ICNN nelvork paranelers are adjusled lo generale nev regions
for LI idenlificalion.
Iuzzy Iogic has leen appIied in delecling Iicense pIales. Aulhors nade sone inluilive ruIes
lo descrile lhe Iicense pIales and gave sone nenlership funclions for fuzzy sels e.g.
lrighl, dark, lrighl and dark sequence, lexlure, yeIIovness lo gel lhe horizonlaI
and verlicaI pIale posilions (Chang el aI., 2OO4). A lechnique lased on exlracls candidale
regions ly finding verlicaI and horizonlaI edges fron vehicIe region had aIso leen proposed
and lhis segnenlalion nelhod is naned as sIiding concenlric vindovs. IinaIIy, vehicIe
Segmenting the License Plate Region Using a Color Model 403

Iicense pIale is verified and delecled ly using HSI coIor nodeI and posilion hislogran,
respecliveIy in (Del el aI., 2OO8a). Irior knovIedge of LI and coIor coIIocalion has leen
used lo Iocale lhe Iicense pIale in lhe inage (Cao el aI., 2OO7) as parl of lhe procedure of
Iocalion and segnenlalion. In (HongIiang & Changping, 2OO4), a hylrid Iicense pIale
IocaIizalion aIgorilhn lased on lhe edge slalislics and norphoIogy for noniloring lhe
highvay lickeling syslen is proposed. This lechnique can le divided inlo four seclions,
vhich are, verlicaI edge deleclion, edge slalislicaI anaIysis, hierarchicaI-lased Iicense pIale
Iocalion, and norphoIogy-lased Iicense pIale exlraclion. The average accuracy of Iocaling
Iicense pIale is an inpressive rale of 99.6. Hovever, inpul inages vere acquired fron a
fixed dislance and viev poinl and lherefore, candidale regions in a specific posilion are
devole priorily as aIready depicled. The Iicense pIale Iocalions in inages are idenlified ly
neans of inlegraled horizonlaI and verlicaI projeclions lhal are scanned using a search
vindov (Huang el aI., 2OO9). Moreover, a characler recovery nelhod is expIoiled lo enhance
lhe success rale. A region-lased Iicense pIale deleclion nelhod has leen presenled in (}ia el
aI., 2OO7), vhich firslIy appIies a nean shifl procedure in spaliaI-range donain lo segnenl a
coIor vehicIe inage in order lo gel candidale regions. According lo lhe slalislicaI anaIysis
perforned for conparison lo olher LI Iike oljecls, LIs adhere lo a unique fealure
conlinalion of reclanguIarily, aspecl ralio, and edge densily. These lhree fealures vere lhen
eslinaled lo candidale regions lo decide vhelher lhese regions inlerprel an LI or nol. A
usuaI faiIure of lhis nelhod is lhe faiIure lo delecl Iicense pIales vhen vehicIe lodies and
lheir Iicense pIale have siniIar coIors. In (}iao el aI., 2OO9), a nelhod for nuIli-slyIe LI
recognilion has leen presenled. This nelhod has inlroduced lhe densily-lased region
groving aIgorilhn for LI Iocalion, lhe skev refinenenl aIgorilhn, lhe nuIli-Iine LI
separalion aIgorilhn, lhe oplinized characler segnenlalion aIgorilhn and lrainalIe
characler recognilion nelhod for characler recognilion. Hough Transforn (HT) for Iine
deleclion has leen proposed on lhe assunplion lhal lhe shape of Iicense pIale has leen
defined ly Iines in (Kanal & Cansen, 1995).
A nodified coIor lexlure-lased nelhod for delecling Iicense pIale in inages has leen
presenled in (Kin el aI., 2OO2). A supporl veclor nachine (SVM) has leen used lo anaIyze
lhe coIor and lexlure properlies of LIs and lo Iocale lheir lounding loxes appIied ly a
conlinuous adaplive nean shifl aIgorilhn (CAMShifl). The conlinalion of CAMShifl and
SVMs produces efficienl LI deleclion as line-consuning coIor lexlure anaIysis for Iess
reIevanl pixeIs is reslricled, Ieaving onIy a snaII parl of lhe inpul inage lo le anaIyzed. In
addilion, finding candidale areas ly using gradienl infornalion, il has leen verified
vhelher il conlains lhe pIale area anong lhe candidales and adjusling lhe loundary of lhe
area ly inlroducing a lenpIale of lhe LI in (Kin el aI., 2OO2). Olher approaches using
nalhenalicaI norphoIogy nelhod lo delecl Iicense pIale area (Marlin el aI., 2OO2) and an
approach for segnenlalion of vehicIe pIales such as edge inage inprovenenl lo delecl a
nunler of car pIales in (Ming el aI., 1996) have aIso leen proposed. The proposed nelhod
in (Nonura el aI., 2OO5) is connilled lo lhe lask of characler segnenlalion, descriling a
norphoIogy-lased adaplive approach for degraded pIale inages.
Moreover, assuning lhal LI regions are deleclalIe even in noisy Iov resoIulion presenled, a
rolusl superresoIulion aIgorilhn for video sequences (Suresh el aI., 2OO7) has leen
proposed lo enhance lhe LI lexl of noving vehicIes vilh pronising resuIls. In (Wang el aI.,
2OO7), a cascade franevork, uliIizing pIale characlerislics and deveIoping fasl one pass
aIgorilhns, has leen used for a reaI-line pIale recognilion syslen.
Pattern Recognition 404

CurrenlIy, sone researchers prefer a hylrid deleclion aIgorilhn, vhere Iicense pIale
Iocalion nelhod lased on corner deleclion, edge deleclion, characlerislics of Iicense shape,
characler's conneclion, and projeclion has leen presenled in (Xu & Zhu, 2OO7), (Zhang el aI.,
2OO7) and (Yang el aI., 2OO6) is anolher nelhod vhich is lased on lhe coIor coIIocalion of lhe
pIale's lackground and characlers conlined vilh lhe pIale's slruclure and lexlure lo Iocale
lhe VLI. In (Zhang el aI., 2OO6), a cascade cIassifier for Iicense pIale deleclion aIgorilhn
using lolh gIolaI slalislicaI fealures and IocaI Haar-Iike fealures is proposed. Using Haar-
Iike fealures nakes cIassifier le invarianl lo lhe lrighlness, coIor, size and posilion of Iicense
pIales. On lhe olher hand, using gIolaI slalislicaI fealures nakes lhe finaI cIassifier sinpIe
and efficienl. Inage enhancenenl and soleI operalor lo exlracl oul verlicaI edges and finaIIy
search pIale region ly a reclanguIar vindov has leen presenled in (Zheng el aI., 2OO5).

3. Specific features of Korean VLP
In lhis seclion, lhe coIor arrangenenls of lhe pIale and oulIine of lhe Korean VLIs lhal are
considered in lhis sludy have leen discussed.

3.1 CoIor arrangement of the pIate
Korean Iicense pIales are veII cIassified as shovn in Iig. 1. Lach slyIe has a differenl pIale
coIor and/or characler coIor. Hovever, in aII, onIy five dislincl coIors Iike vhile, lIack,
green, yeIIov, and deep lIue are used in lhese Iicense pIales. Il is vorlh paying allenlion lo
lhree differenl pIale coIors vhiIe searching for LI in an inpul inage. Olher lypes of vehicIes,
such as dipIonalic cars and niIilary vehicIes, are nol addressed since lhey are rareIy seen.
CoIor arrangenenls for lhe Korean VLIs are shovn in TalIe 1.


Iig. 1. OulIine of lhe Korean Iicense pIale

3.2 OutIine of the Korean VLP
Slandard LI conlains Korean aIphalels and nunlers vhich are shovn in Iig. 1. Iev LIs
conlain Korean aIphalels and nunlers in lvo rovs, in fulure lhese kinds of LIs are lo le
converled inlo singIe-rov lypes. Where pIale coIor is vhile and characler coIor is lIack, lhey
conlain seven aIphanuneric characlers vrillen in a singIe Iine. In Iig. 1, vhere pIale coIor is
green and characler coIor is vhile, lhey conlain Korean LI in lvo rovs. The upper rov
consisls of lvo snaII Korean characlers of region nane foIIoved ly one or lvo nunlers of
cIass code or lvo nunlers and one Korean characler. The Iover rov is one Korean characler
and four lig nunlers or onIy four lig nunlers lo indicale lhe usage and seriaI nunler,
respecliveIy. When pIale coIor is yeIIov and characler coIor is lIack, sone LIs conlain aII
Segmenting the License Plate Region Using a Color Model 405

aIphanuneric characlers vrillen in a singIe Iine and anolher lype of yeIIov LI is found lhal
conlain Korean LI in lvo rovs. The upper rov consisls of lvo snaII Korean characlers of
region nane foIIoved ly one or lvo nunlers of cIass code. The Iover rov conlains one
Korean characler and four lig nunlers lo indicale lhe usage and seriaI nunler,
respecliveIy.

VehicIe lype IIale coIor Characler coIor
Irivale aulonoliIe
While Iack
Creen While
Taxi, lruck, and lus YeIIov Deep lIue
Covernnenl vehicIe YeIIov Iack
TalIe 1. SlyIes of Iicense pIales

4. Proposed LP detection framework
In lhe aulhor's previous vork (Del & }o, 2OO8l), HSI coIor lased vehicIe Iicense pIale
deleclion nelhod vas presenled. We propose in lhis chapler an enhanced version of lhe
franevork for VLID as shovn in Iig. 2. Like lhe lradilionaI LI deleclion nelhod, aulonalic
focus and vhile laIancing of canera oflen cause lhe changing iIIuninalion. To overcone
lhis prolIen, ve propose an adaplive LI deleclion nelhod for delecling vhile Iicense pIale
pixeIs, ve use il in lhe case of reaIIy high- or Iov-iIIuninalion condilion as shovn in Iig. 4.
And aIso dislinguish vilh lhe lradilionaI LI deleclion nelhod, as Iicense pIales can appear
al nany differenl angIes lo lhe canera's oplicaI axis, each reclanguIar candidale region is
rolaled unliI lhey are aII aIigned in lhe sane vay lefore lhe candidale deconposilion. The
proposed franevork can efficienlIy delernine and adjusl lhe rolaled pIale as shovn in Iig.
8. Measurenenls such as cenler of area and lhe Ieasl second nonenl are enpIoyed lo soIve
lhe rolalion adjuslnenl prolIen. The Ieasl second nonenl provides lhe principaI axis as lhe
orienlalion vilh lhe candidale oljecl. CeneraI franevork for delecling VLI region is shovn
in Iig. 2. In lhe proposed franevork, deleclion is lased on coIor properlies of LI, shape-
lased verificalion and posilion hislogran.

5. VehicIe Iicense pIate detecting moduIe
The VLID sequence is shovn in Iig. 2, vhich is proposed in lhis paper, consisls of four
dislincl parls. The firsl one deaIs vilh, ly using HSI coIor nodeI, lhe deleclion of lhe
candidale region, i.e., lhe Iicense pIale. The second parl aIIovs procedures for refining
candidale region ly using IaleIing and fiIlering. According lo differenl coIored LI lhese
candidale regions nay incIude reclanguIar LI regions, geonelricaI properlies of LI such as
area, lounding lox, and aspecl ralio are lhen used for cIassificalion. The lhird parl incIudes
operalions for delernining lhe angIe of lhe candidale rolalion adjuslnenl. Measurenenls
such as cenler of area and lhe Ieasl second nonenl are enpIoyed lo soIve lhe rolalion
adjuslnenl. The fourlh parl incIudes perfornances for candidale's deconposilion and
finaIIy, lhe deconposilion of candidale region vhich conlains predelernined LI
Pattern Recognition 406

aIphanuneric characler ly using posilion in lhe hislogran lo verify and delecl vehicIe
Iicense pIale (VLI) region.

Iig. 2. VehicIe Iicense pIale deleclion franevork

5.1 Segmenting coIor
In lhe proposed nelhod, inpul vehicIe inages are converled inlo HSI coIor inages. Then lhe
candidale regions are found ly using HSI coIor nodeI on lhe lasis of using hue, saluralion
and/or inlensily. Many appIicalions use lhe HSI coIor nodeI. Machine vision uses HSI coIor
space in idenlifying lhe coIor of differenl oljecls.
The RC coIor nodeI consisls of lhe lhree addilive prinaries: red, green, and lIue. SpeclraI
conponenls of lhese coIors conline addiliveIy lo produce a resuIlanl coIor. TypicaIIy, HSI
coIors are nol descriled on lhe lasis of percenlages of prinary coIors, lul ralher ly lheir
hue, saluralion and inlensily. The saluralion is lhe "pureness" of lhe coIor, lhe hue is lhe
coIor ilseIf and inlensily descriles lhe lrighlness of lhe coIor. The HSI coIor nodeI separales
aII lhe coIor infornalion, descriled ly hue and saluralion, fron lhe inlensily conponenl.
The HSI coIor nodeI is lased on coIor descriplions lhal are nore naluraI lo hunans and
hence can provide an ideaI looI for inage processing aIgorilhns. The HSI coIor space is
represenled ly lhe dianond, as shovn in Iigure 3. The hue H is represenled as angIe 0,
varying fron 0

lo 360. Adjusling lhe hue viII vary lhe coIor fron red al 0, lhrough yeIIov
al 60, green al 120, lIue al 240 and lack lo red al 360. Saluralion S corresponds lo lhe
radius, varying fron 0 lo 1. When S =O, coIor is a gray vaIue of inlensily 1. When S =1, coIor
is on lhe loundary of lop cone lase. Inlensilies | vary aIong Z axis vilh O leing lIack and 1
leing vhile.

Segmenting the License Plate Region Using a Color Model 407


Iig. 3. The HSI coIor space

The lransforn fron (R, C, ) lo (H, S, I) in (Unlaugh, 1998) is
( ) ( )
( ) ( )( )
( )
-1
1
2
1
2
cos
2
3
1 - min , ,



3
- -

- - -


6 5 * %
5 * %
5 * %
5 * 5 %
+
5 * 5 % * %
,
=
=
+ +
+ +
=
( +

(
+
(







(1)

IIale coIor infornalion is used lo delecl candidale regions in our experinenls, and shape
properlies of LI aIIov reducing nunler of LI-Iike candidales. One of lhe connon vays of
coIor-lased vehicIe Iicense pIale deleclion can le fornaIized as foIIovs:
( , ) ; ( , ) ; ( , )
( , ) - ( , ) ; ( , ) - ( , )
5 * %
5* 5%
5 [ \ * [ \ % [ \
5 [ \ * [ \ 5 [ \ % [ \
o o o
| |
> > >
> >

(2)
(3)
vhere R, G and 8 are red, green and lIue conponenls of x y inage. o and | are
predefined coefficienls. Lqualion (2) sels up Iinilalions for lhe nininaI vaIues of pixeI
conponenls. Lqualion (3) fornaIizes dependencies lelveen pixeI conponenls for LI.
CeneraIIy, connon vay of using coIor-lased vehicIe Iicense pIale deleclion is lased on lvo
lypes of reslriclions: firsl, reslriclion is lased on Lps. (2) and (3). Il provides good resuIls in
good Iighling condilions. Hovever, il is nol good for Iov-conlrasl inages. IixeI leIongs lo
green and yeIIov LI, respecliveIy Iike foIIoving Lqs. (4) and (5)
Pattern Recognition 408

{ } { } 1, ( , ) 0.85 ( , ) & ( , ) 0.90 ( , )
0,
JUHHQ
5 [ \ * [ \ % [ \ * [ \
E
RWKHUZLVH

( s s

{ } { } 1, ( , ) 0.90 ( , ) & ( , ) 0.80 ( , )
0,
\HOORZ
% [ \ 5 [ \ % [ \ * [ \
E
RWKHUZLVH

( s s

(4)


(5)
vhere |
grccn
and |
qc||cu
are green and yeIIov candidale linary nasks. The second reslriclion
is lased on Lqs. (4) and (5), and a lhreshoId vaIue is laken heurislicaIIy. Il provides slalIe
resuIl vhereas in lad Iighling condilion il is loo sensilive.
In lhis proposed nelhod, LI deleclion is lased on ils coIor properlies, naneIy nean and
slandard devialion vaIues of hue. Ior deleclion of green and yeIIov LI pixeIs, hue
paraneler of HSI coIor is used in our experinenl. To delecl vhile LI pixeIs hue vaIue is
neaningIess, hence onIy saluralion and inlensily paranelers are inporlanl for lhis case. To
eslinale lhese properlies, ve used 3O inages of LI laken under differenl Iighling and
vealher condilions. Afler lraining fron lhose sanpIe dala, lhe nean and slandard devialion
vaIues of hue are conpuled for deleclion of green and yeIIov LI pixeIs. Delecling vhile
Iicense pIale pixeIs, lhe nean and slandard devialion vaIues of saluralion and inlensily are
conpuled lo delecl green, yeIIov and vhile LI fron vehicIe inages. Ior deleclion of green
and yeIIov LI pixeIs, lhe linarizalion process can le fornuIaled as foIIovs:
{ } { }
{ }
1, - ( , ) & ( , ) 0.08
& 0.05 ( , ) 0.95
0,
+ + + +
JUHHQ
+ [ \ 6 [ \
E , [ \
RWKHUZLVH
o o

s s + >

( = s s

{ } { }
{ }
1, - ( , ) & ( , ) 0.12
& 0.20 ( , ) 0.80
0,
+ + + +
\HOORZ
+ [ \ 6 [ \
E , [ \
RWKHUZLVH
o o

s s + >

( = s s

(6)



(7)

vhere H(x, q), S(x, q), and |(x, q) are hue, saluralion and inlensily conponenls of xlh, qlh
pixeI, respecliveIy.
H
and o
H
are nean hue and hue slandard devialion vaIues for green and
yeIIov LI of sanpIe dala, respecliveIy.
Hovever, lhe aulonalic focus and vhile laIancing of canera oflen cause lhe changing
iIIuninalion. Our proposed LI deleclion nelhod can vork veII in nornaI iIIuninalion
condilion, lul il seens nol good enough lo vork in lad iIIuninalion condilions. To
overcone lhis prolIen, ve use an adaplive LI deleclion nelhod, ve use il in lhe case of
reaIIy high- or Iov-iIIuninalion condilion.
Ior nornaI, Iov- and high-iIIuninalion condilions of vhile Iicense pIale pixeIs, lhe
linarizalion process can le fornuIaled as foIIovs, respecliveIy:
( ) { } { }
( )
1, ( , ) & ( , ) 0.25

0,
6 6 , ,
ZKLWH Q
6 [ \ , [ \
E
RWKHUZLVH
o o

(
s + > +

( ) { } { }
( )
1, ( , ) & ( , ) - 0.33

0,
6 6 , ,
ZKLWH O
6 [ \ , [ \
E
RWKHUZLVH
o o

(
s + >

(8)



(9)

Segmenting the License Plate Region Using a Color Model 409

( ) { } { }
( )
1, ( , ) & ( , ) 0.50

0,
6 6 , ,
ZKLWH K
6 [ \ , [ \
E
RWKHUZLVH
o o

(
s + > +

(1O)


vhere S(x, q) and |(x, q) are saluralion and inlensily conponenls of xlh, qlh pixeI
respecliveIy.
S
and
|
are nean vaIues for saluralion and inlensily, o
S
, o
|
are slandard
devialion vaIues for saluralion, inlensilies of vhile LI of sanpIe dala, respecliveIy. |
uni|c(n)
,
|
uni|c(|)
and |
uni|c(n)
are vhile candidale linary nasks. An LI inage and ils coIor segnenlalion
resuIls are depicled in Iig. 4(a) - (c) (green, yeIIov and vhile lack ground LI), respecliveIy.


Iig. 4. An LI inage (|cf|) and ils coIor segnenlalion resuIls (rign|) using HSI coIor nodeI

CoIor segnenlalion paranelers are very sensilive in order lo delecl as nany candidales as
possilIe. AII faIse candidales viII le fiIlered oul on lhe nexl slages. According lo lhe prior
knovIedge of vehicIe LI inspeclion, aII Iicense pIales nusl le reclanguIar in shape and have
lhe dinensions and have aII aIphanuneric characlers vrillen in one or lvo rovs, in LI
region. Afler lhe segnenlalion, lhere nay sliII exisl noises in lhe inage and lhal is nol ideaI.
These noises have nany lypes, such as snaII hoIes or/and luIges of lhe largel candidale
regions. The prolIen nay le resoIved ly using nalhenalicaI norphoIogy processing
nelhod. MalhenalicaI norphoIogy is used as a polenl looI for inage anaIysis vhich is
lased on shapes in lhe inage, nol pixeI inlensilies. The lvo principaI norphoIogicaI
operalions are diIalion and erosion. DiIalion aIIovs oljecls lo expand and erosion shrinks
oljecls ly elching avay (eroding) lheir loundaries. These operalions can le cuslonized ly
lhe proper seIeclion of lhe slrucluring eIenenl, vhich delernines exaclIy hov lhe oljecls
viII le diIaled or eroded. DiIalion and erosion are conlined inlo olher lvo operalions:
opening and cIosing. In lhis parl of lhe appIicalion, ve use lhe cIosing operalion vhich is
Pattern Recognition 410

diIalion foIIoved ly erosion lo fiII in hoIes and gaps snaIIer lhan lhe slrucluring eIenenl on
lhe pIale inage. RenovaI of lhose hoIes pIays an inporlanl roIe in caIcuIaling lounding lox
region. InpIenenlalion of norphoIogicaI cIosing operalion is depicled in Iig. 5(c) - 7(c),
respecliveIy.

5.2 LabeIing and fiItering
Afler lhe candidale regions are ollained ly appIying coIor segnenlalion, fealures of each
region are lo le exlracled in order lo correclIy differenliale lhe LI regions fron olhers. Nexl
slep of proposed aIgorilhn is IaleIing lhe connecled conponenls. In lhe proposed nelhod,
a recursive aIgorilhn is inpIenenled for connecled conponenl IaleIing operalion.
Recursive aIgorilhn (Shapiro el aI., 2OO1) vorks on one conponenl al a line, lul can nove
aII over lhe inage. In lhis slep ve exlracl candidale regions vhich nay incIude LI regions
fron lhe linary nask ollained in lhe previous slep. During lhis slep, nain geonelricaI
properlies of LI candidale such as area, lounding lox, and aspecl ralio are conpuled.
IoIIoving lhe successfuI connecled conponenl IaleIing operalion in inage, neasurenenls
such as lhe area, lhe lounding lox and lhe aspecl ralio for every linary oljecl in lhe inage
are perforned.
A lounding lox is a reclangIe vhose horizonlaI and verlicaI sides encIose lhe region and
louch ils lopnosl, lollonnosl, Ieflnosl, and righlnosl poinls. ReclanguIarily is defined as
lhe ralio of lhe area of candidale oljecl's MLR (nininun encIosing reclangIe) and lhe area
of lhe oljecl. Here, lhe area is neasured in pixeIs and indicales lhe reIalive size of lhe oljecl.
The aspecl ralio (aIso caIIed eIongalion or eccenlricily), is defined ly lhe ralio of lhe
lounding lox of an oljecl. This can le found ly scanning lhe inage and lhe nininun and
naxinun vaIues on lhe rov and lhe coIunns, vhere lhe oljecl Iies. This ralio is defined ly

max min
max min
1
1
$
F F
U U

+
=
+


(11)

vhere c and r indicale coIunns and rov, respecliveIy. Oljecls vhich salisfy
A
(aspecl ralio)
lounds 1 lo 3 for green, 1 lo 2 for yeIIov, and 1 lo 6 for vhile LIs are considered as
candidale regions. These paranelers are used for fiIlering operalion lo eIininale LI-Iike
oljecls fron candidale Iisl. IiIlering operalion is done on geonelricaI properlies of LI
regions. Iigs. 5 - 7 iIIuslrale lhe sleps for Iicense pIale segnenlalion: (a) an LI inage, (l)
coIor segnenlalion resuIl, (c) inpIenenlalion of norphoIogicaI cIosing operalion for
renoving snaII hoIes in candidale region, (d) delecled candidale afler fiIlering, and (e)
candidale region deleclion. The nosl inporlanl LI-paranelers are grouped in TalIe 2.

IiIlering paraneler CR (green) CR (yeIIov) CR (vhile)
ounding lox |O.6, 1.Oj |O.7, 1.Oj |O.7, 1.Oj
Aspecl ralio |1.O, 3.Oj |1.O, 2.Oj |1.O, 6.Oj
IossilIe shapes ReclangIe
TalIe 2. IiIlering properlies


Segmenting the License Plate Region Using a Color Model 411

Here CR indicales candidale region.

Iig. 5. IIIuslralion of Iicense pIale segnenlalion: (a) an LI inage in a nighl line, (l) coIor
segnenlalion resuIl, (c) inpIenenlalion of norphoIogicaI cIosing operalion for renoving
snaII hoIes in candidale region, (d) delecled candidale afler fiIlering, and (e) candidale
region deleclion

Iig. 6. IIIuslralion of Iicense pIale segnenlalion: (a) an LI inage in a slrong sunshine, (l)
coIor segnenlalion resuIl, (c) inpIenenlalion of norphoIogicaI cIosing operalion for
renoving snaII hoIes in candidale region, (d) delecled candidale afler fiIlering, and (e)
candidale region deleclion

Iig. 7. IIIuslralion of Iicense pIale segnenlalion: (a) an LI inage in a slrong sunshine
refIecled ly vehicIe nirror and aIso a Iighl posl Iocaled in fronl of vehicIe, (l) coIor
segnenlalion resuIl, (c) inpIenenlalion of norphoIogicaI cIosing operalion for renoving
snaII hoIes in candidale region, (d) delecled candidale afler fiIlering, and (e) candidale
region deleclion

5.3 Determining the angIe of the candidate - rotation adjustment
As Iicense pIales can appear al nany differenl angIes (in our experinenl is nore rolusl
vhen LI is rolaled fron -15 lo +15 degree) lo lhe canera's oplicaI axis, each reclanguIar
candidale regions is rolaled unliI lhey are aII aIigned in lhe sane vay lefore lhe candidale
deconposilion. IoIIoving lhe successfuI fiIlering operalion in inage, neasurenenls such as
Pattern Recognition 412

cenler of area and lhe axis of Ieasl second nonenl are enpIoyed lo soIve lhe rolalion
adjuslnenl prolIen.
The cenler of area (cenlroid), is lhe nidpoinl aIong each rov and coIunn axis corresponding
lo lhe niddIe lased on lhe spaliaI dislrilulion vilhin candidale oljecl. This fealure used
lo Iocale an oljecl in lhe 2D inage pIan is defined ly lhe pair
( )
, :
L L
U F

( )
( )
1 1
0 0
1 1
0 0
1
,
1
,
1 1
L L
L
U F
1 1
L L
L
U F
U U , U F
$
F F , U F
$

= =

= =
=
=
__
__


(12)


(13)
vhere
L
U and
L
F indicales rov and coIunn coordinale of lhe cenler of area for lhe ilh oljecl.
The area,
L
$ , is neasured in pixeIs and indicales lhe reIalive size of lhe oljecl.
The Ieasl second nonenl provides lhe principaI axis as lhe orienlalion vilh lhe candidale
oljecl. Ior gelling principaI axis of delecled candidale region, ve conpule cenlraI nonenls
of delecled candidale region. The cenlraI nonenls are defined as

( ) ( )
( )
1 1
0 0
,
1 1
S T
ST
U F
U U F F , U F

= =
=
__


(14)
We appIy lhis resuIl lo ollain a direclion of principaI axis ly cenlroid of delecled candidale
region. AngIe of principaI axis nonenls is ollained as

11
20 02
1 2
arctan
2

u

| |
=
|
|

\ .


(15)
vhere u denoles an angIe lelveen lasis horizonlaI coordinale and principaI axis of region.
Iigure 8 porlrays a sequence of successfuI Iicense pIale idenlificalion.


Iig. 8. IIIuslralion of Iicense pIale segnenlalion: (a) an LI inage (l) delecled candidale afler
fiIlering (c) principaI axis (d) rolalion adjuslnenl, and (e) exlracled candidale

5.4 Decomposing candidates
Infornalion exlracled fron inage and inlensily hislograns pIays a lasic roIe in inage
processing, in areas such as enhancenenl, segnenlalion, and descriplion. In lhis seclion,
Segmenting the License Plate Region Using a Color Model 413

verificalion and deleclion of lhe VLI region as veII as characler segnenlalion are
considered and discussed in lhis sludy. The aIgorilhn schene for candidale deconposilion
is shovn in Iig. 9.
Once lhe candidale area is linarized, lhe nexl slep is lo exlracl lhe infornalion. Al firsl,
regions vilhoul inleresl such as lorder or sone snaII noisy regions are eIininaled, lhe
checking is nade ly heighl conparison vilh olher pIale characlers heighl. IoIIoving
procedure is perforned vhen LI coIor is green and yeIIov: firsl ve proceed ly perforning
horizonlaI posilion in lhe hislogran, lvo oljecls are found vhere each oljecl corresponds
vilh one rov. Then lhe rovs are isoIaled and processed separaleIy. As nenlioned lefore in
Secl. 3, lvo lypes of pIale are considered.


Iig. 9. AIgorilhn schene for candidale deconposilion

Irocessing of lhe upper rov: firsl fiIler phase is perforned lo eIininale lhe regions vilhoul
inleresl. Then verlicaI posilion hislogran is processed. The upper rov aIso has lvo differenl
lypes as ve nenlioned in Secl. 3. As il can le olserved, usuaIIy in lhe upper rov ve can
find lvo pIale-fixing dols as shovn in Iig. 1. The righl pIale-fixing dol does nol even appear
in lhe linarizalion process due lo lhe facl lhal il is prinled green. The Iefl pIale-fixing dol is
aIso eIininaled. The checking is nade ly heighl conparison. Iron lhe verlicaI posilion in
lhe hislogran ve can find isoIaled aIphanuneric characlers.

Pattern Recognition 414


Iig. 1O. LxanpIe inages: (a) differenl iIIuninalions, (l) conpIex scenes, (c) various
environnenls, and (d) danaged Iicense pIales

Irocessing of lhe Iover rov: firsl fiIler phase is perforned lo eIininale lhe regions vilhoul
inleresl. Then verlicaI posilion hislogran is perforned and fron lhe verlicaI posilion
hislogran lhe aIphanuneric characlers are isoIaled.
According lo lhe prior knovIedge of vehicIe LI inspeclion, aII vhile LIs conlain seven
aIphanuneric characlers as veII as vrillen in a singIe rov. The foIIoving procedure is
perforned for characler segnenlalion: Afler eIininaling lorder area, verlicaI posilion in lhe
hislogran is perforned for segnenling predelernined aIphanuneric characlers. As il can
le olserved, usuaIIy ve can find aIso lvo pIale-fixing dols in upper area of pIale region.
The righl pIale fixing dol or lolh pIale-fixing dols do nol even appear in lhe linarizalion
Segmenting the License Plate Region Using a Color Model 415

process due lo lhe facl lhal il is prinled vhile. The Iefl pIale-fixing dol is aIso eIininaled,
lhis checking is nade ly heighl conparison.
Iigure 11 shovs lhe foIIoving sleps for verifying predelernined aIphanuneric characlers
(vhile lack ground LI): (a) exlracling candidale region, (l) verlicaI posilion hislogran vilh
LI lorder, (c) horizonlaI posilion hislogran vilh LI lorder, (d) horizonlaI posilion
hislogran vilhoul LI lorder,(e) viev of nornaIizalion candidale region afler renoving
lorder and noisy area, (f) verlicaI posilion hislogran (seven peaks for predelernined seven
aIphanuneric characlers in LI region), and (g) characler exlraclion.

6. ExperimentaI resuIts

AII experinenls have leen done on Ienliun-IV 2.4 CHz vilh 1O24 M RAM under MalIal
environnenl. In lhe experinenls, 15O inages vere used lhe size is 64O 48O pixeIs, sone
inages vhich are shovn in Iig. 1O. The inages are laken fron (a) differenl iIIuninalions
(nighl line, slrong sunshine, and shadov), (l) conpIex scenes vhere severaI oljecls such as
lrees, Iighl posl in fronl of vehicIes, (c) various environnenls in canpus parking, access
areas and nore lhan one Iicense pIales in lhe sane inage, and (d) danaged LI as lenl or
oId. They vere laken in dislance fron 2 up lo 8 n and lhe canera vas focused in lhe pIale
region. Under lhese condilions, success of LI deleclion has reached lo nore lhan 94.
ResuIls of candidale region deleclion are shovn in TalIe 3.

Inage group TolaI inages Delecled LIs Success rale ()
Differenl iIIuninalions 57 53 92.98
ConpIex scenes 15 14 93.34
Various environnanls 73 69 94.52
Danaged LI 5 5 1OO
TolaI 15O 141 94
TalIe 3. Deleclion resuIls

A connon dravlack of lhe proposed VLID syslen is lhe faiIure lo delecl lhe loundaries or
lorder of Iicense pIales. This occurs vhen vehicIe lodies and lheir Iicense pIale possess
siniIar coIors. The average conpulalionaI line for lhe coIor segnenlalion and fiIlering
operalions of lhe proposed nelhod are shovn in TalIe 4.

Slage Avg line (s) Sld. devialion (s)
CoIor segnenlalion O.16 O.O7
IiIlering O.O7 O.O2
TalIe 4. Average conpulalion line
Pattern Recognition 416


Iig. 11. Sleps for verify predelernined aIphanuneric characlers (vhile lack ground LI): (a)
exlracling candidale region, (l) verlicaI posilion hislogran vilh LI lorder, (c) horizonlaI
posilion hislogran vilh LI lorder, (d) horizonlaI posilion hislogran vilhoul LI lorder,(e)
viev of nornaIizalion candidale region afler renoving lorder and noisy area, (f) verlicaI
posilion hislogran (seven peaks for predelernined seven aIphanuneric characlers in LI
region), and (g) characler exlraclion.

7. ConcIusion

In concIusion, a nev nelhod is adopled in lhis paper lo seIecl slalislicaI lhreshoId vaIue in
HSI coIor nodeI. In lhe proposed nelhod candidale regions are found ly using HSI coIor
nodeI. These candidale regions nay incIude LI regions, geonelricaI properlies of LI are
lhen used for cIassificalion. The proposed nelhod is alIe lo deaI vilh candidale regions
under independenl orienlalion and scaIe of lhe pIale. IinaIIy, VLI regions conlaining
predelernined LI aIphanuneric characler are verified and delecled ly using posilion
hislogran. CoIor arrangenenl and predelernined LI aIphanuneric characler of lhe Korean
Iicense pIale are inporlanl fealures for verificalion and deleclion of Iicense pIale regions.
WhiIe conducling lhe experinenls, differenl iIIuninalion condilions and varied dislances
lelveen vehicIe and canera oflen occurred. In lhal case, lhe resuIl lhal has leen confirned
is very nuch effeclive vhen lhe proposed approach is used. Hovever, lhe proposed nelhod
is sensilive vhen vehicIe lodies and lheir Iicense pIales possess siniIar coIors. We Ieave
lhese issues lo le considered in fulure sludies.
Segmenting the License Plate Region Using a Color Model 417

8. AcknowIedgments
The aulhors vouId Iike lo lhank lo UIsan MelropoIilan Cily, MKL and MLST vhich have
supporled lhis research in parl lhrough lhe NARC, lhe Hunan Resource Training projecl
for regionaI innovalion lhrough KOTLI, lhe Hunan Resource DeveIopnenl Irogran for
Convergence Rolol SpeciaIisls lhrough KIAT and posl K21 al Universily of UIsan. AIso,
ve express speciaI lhanks lo IITA for lheir inlernalionaI graduale sludenls schoIarship.

9. References

AnagnoslopouIos, C., AnagnoslopouIos, I., Lounos, V., & Kayafas, L. (2OO8). License pIale-
recognilion fron sliII inages and video sequences: a survey. ILLL Trans. on InleII.
Transp. Sysls., VoI. 9, No. 3, Seplenler 2OO8, pp. 377-391
AnagnoslopouIos, C., AnagnoslopouIos, I., Lounos, V., & Kayafas, L. (2OO6). A Iicense
pIale-recognilion aIgorilhn for inleIIigenl lransporlalion syslen appIicalion, ILLL
Trans. on InleII. Transp. Sysls., VoI. 7, No. 3, Seplenler 2OO6, pp. 377-392
Cano, }., & Ierez-Corles, }. C. (2OO3). VehicIe Iicense pIale segnenlalion in naluraI inages.
In: Leclure noles on conpuler science, VoI. 2652, pp 142-149
Chacon, M. I., & Zinnernan, A. (2OO3). License pIale Iocalion lased on a dynanic ICNN
schene. In: Iroceedings of lhe Inl. }oinl Conf. on NeuraI Nelv., VoI. 2, pp 1195-
12OO
Chang, S. L., Chen, L. S., Chung, Y. C., & Chen, S. W. (2OO4). Aulonalic Iicense pIale
recognilion. ILLL Trans. InleII. Transp. Sysl., VoI. 5, No. 1, pp 42-53
Del, K., Chae, H. U., & }o, K. H. (2OO8a). VehicIe Iicense pIale deleclion nelhod lased on
sIiding concenlric vindovs and hislogran. }ournaI of Conpulers, VoI. 4, No. 8,
Augusl 2OO9, pp 771-777
Del, K. & }o, K. H. (2OO8l). HSI coIor lased vehicIe Iicense deleclion. In: Iroceedings of lhe
ILLL ICCAS, pp 687-691, Ocloler 2OO8, SeouI, Korea
Cao, Q., Wang, X. & Xie, C.(2OO7). License pIale recognilion lased on prior knovIedge. In:
Iroceedings of lhe ILLL Inl. Conf. on Aulonalion and Logislics, pp 2964-2968
HongIiang, . & Changping, L. (2OO4). A hylrid Iicense pIale exlraclion nelhod lased on
edge slalislics and norphoIogy. In: Iroceedings of lhe ILLL ICIR, pp 831-834
Huang, Y. I., Chen, C. H., Chang, Y. T. & Sandnes, I. L. (2OO9). An inleIIigenl slralegy for
checking lhe annuaI inspeclion slalus of nolorcycIes lased on Iicense pIale
recognilion, Lxperl Sysl. vilh AppI., VoI. 36, No. 5, 2OO9, pp 926O-9267
}ia, W., Zhang, H. & He, X. (2OO7). Region-lased Iicense pIale deleclion. }ournaI of Nelvork
and Conpuler AppI., VoI. 3O, No. 4, pp 1324-1333
}iao, }., Ye, Q., & Huang, Q. (2OO9). A configuralIe nelhod for nuIli-slyIe Iicense pIale
recognilion, Iallern Recognil., VoI. 42, No. 3, pp 358-369
Kanal, V. & Canesan, S. (1995). An efficienl inpIenenlalion of lhe hough lransforn for
delecling vehicIe Iicense pIales using DSI'S. In: Iroceedings of lhe ILLL Inl. Conf.
on RTAS, pp 58-59
Kin, K. I., }ung, K., & Kin, }. H. (2OO2). CoIor lexlure-lased oljecl deleclion: an appIicalion
lo Iicense pIale IocaIizalion. In: Ieclure noles on conpuler science, VoI. 2388, pp
293-3O9
Pattern Recognition 418

Kin, S., Kin, D., Ryu, Y., & Kin, C. (2OO2). A rolusl Iicense pIale exlraclion nelhod under
conpIex inage condilions. In: Iroceedings of lhe ILLL Inl. Conf. on ICIR, pp 216-
219
Marlin, I., Carcia, M. & AIla, }. L. (2OO2). Nev nelhods for aulonalic reading of VLI's
(VehicIe License IIales). In: Iroceedings of lhe IASTLD Inl. Conf. on SIIRA
Malas, }., & Zinnernann, K. (2OO5). Unconslrained Iicence pIale and lexl IocaIizalion and
recognilion. In: Iroceedings of lhe ILLL Inl. Conf. on InleII. Transp. Sysl.
Ming, C. H., Harvey, A. L., & DaneIulli, I. (1996). Car nunler pIale's deleclion vilh edge
inage inprovenenl. In: Iroceedings of lhe Inl. Synp. on SignaI process. and ils
AppI., VoI. 2, pp 597-6OO
Nonura, S., Yananaka, K., Kalai, O., Kavakani, H., & Shiose T. (2OO5). A noveI adaplive
norphoIogicaI approach for degraded characler inage segnenlalion, Iallern
Recognil, VoI. 38, No. 11, pp. 1961-1975
Suresh, K. V., Mahesh, K. C., & RajagopaIan, A. N. (2OO7). SuperresoIulion of Iicense pIales
in reaI lraffic videos. ILLL Trans. InleII. Transp. Sysl., VoI 8, pp 321-331
Wang, S. Z. & Lee, H. }. (2OO7). A cascade franevork for a reaI-line slalislicaI pIale
recognilion syslen. ILLL Trans. Inf. Iorensics Securily, VoI. 2, pp 267-282
Xu, Z., & Zhu, H. (2OO7). An efficienl nelhod of Iocaling vehicIe Iicense pIale," In:
Iroceedings of lhe ILLL Inl. Conf. on ICNC, pp 18O-183
Yang, Y. Q., ai, }., Tian, R. L. & Liu, N. (2OO6). Research of vehicIe Iicense pIale Iocalion
aIgorilhn lased on coIor fealures and pIale processions. In: Leclure noles on
conpuler science, voI 393O, pp 1O77-1O85
Zhang, C., Sun, C., Chen, D. & Zhao, T. (2OO7). A rapid Iocaling nelhod of vehicIe Iicense
pIale lased on characlerislics of characlers conneclion and projeclion. In:
Iroceedings of lhe ILLL Inl. Conf. on InduslriaI and AppI., pp 2546~2549
Zhang, H., }ia, W., He, X. & Wu, Q. (2OO6). Learning-lased Iicense pIale deleclion using
gIolaI and IocaI fealures. In: Iroceedings of lhe ILLL ICIR, pp 11O2-11O5
Zheng, D., Zhao, Y., & Wang, }. (2OO5). An efficienl nelhod of Iicense pIale Iocalion. Iallern
Recognil Lell., VoI 26. No. 15, pp 2431-2438
Unlaugh, S. L. (1998) Conpuler Inaging : digilaI inage anaIysis and processing. CRC
Iress, ISN O-8493-2919-1, pp. 45-46
Shapiro, L. C. & Slocknan, C. C. (2OO1). Conpuler vision. Irenlice-HaII, ISN O-13-O3O796-3
Nev }ersey