Proc. VIIth Digital Image Computing: Techniques and Applications, Sun C., Talbot H., Ourselin S.

and Adriaansen T. (Eds.), 10-12 Dec. 2003, Sydney

AutomaticExtractionofSignaturesfromBank
ChequesandotherDocuments
VamsiKrishnaMadasu*,Mohd.HafizuddinMohd.Yusof,M.Hanmandlu,Kurt
Kubik*

*IntelligentReal-TimeImagingandSensinggroup,SchoolofInformationTechnologyand
ElectricalEngineering,UniversityofQueensland,QLD4072,Australia.
{madasu@itee.uq.edu.au,kubik@itee.uq.edu.au}

FacultyofInformationTechnology,MultimediaUniversity,Cyberjaya
64100SelangorD.E.,Malaysia.
hafizuddin.yusof@mmu.edu.my

Dept.ofElectricalEngineering,I.I.T.Delhi,HauzKhas,
NewDelhi110016,India.
mhmandlu@ee.iitd.ernet.in

Abstract:Aninnovativeapproachforextractingsignaturesfrombankcheque
images and other documents is proposed based on the integration of the crop
method with the sliding window technique. The idea is to estimate the
approximate area in which the signature lies using the sliding window
technique.Inthisapproach,awindowofadaptableheightandwidthismoved
overtheimage;onepixelatatimeandthedensityofpixelswithinthewindow
iscalculated.Thisdensityisthenusedtofindtheentropy,whichinturnhelps
fittheboxthatcansegmentthesignature.Thesignaturesthusextractedarethen
fedtoaknownfuzzybasedoff-linesignatureverificationandforgerydetection
system. The proposed method has been applied with almost 100% success on
severalbankchequesfromIndia,MalaysiaandAustralia.Signatureextraction
hasalsobeenshownontwotypicaltypesofdocumentswhichhavevariedand
noisybackgrounds.

Keywords: Bank cheque processing, Signature extraction, Sliding window

method,Entropy

Introduction

Automatic extraction of user entered components from bank cheques and other
document forms has been the prime focus of researchers concerned with document
analysis and recognition for the past one decade. Bank cheques and financial
documentsinpaperformatarestillinenormousdemandinspiteoftheoverallrapid
emergence of e-commerce and online banking. Fraud committed in cheques is also
growingatanequallyalarmingratewithconsequentloss[1].TheAmericanBanker
has projected that check fraud will grow by 25 percent annually in coming years.
AccordingtotheAmericanBankersAssociation's(ABA)1998CheckFraudSurvey,
financialinstitutionsaloneincurred$512.3millionincheckfraudlosses.Whenlosses
toallbusinesseswerefactoredin,thatfigurerosetomorethan$13billion,according
toa1997articleintheSt.LouisBusinessJournal.Inthispaper,wetrytoaddressthe

591

Proc. VIIth Digital Image Computing: Techniques and Applications, Sun C., Talbot H., Ourselin S. and Adriaansen T. (Eds.), 10-12 Dec. 2003, Sydney

problemofchequefraudbytryingtodevelopanewbankchequeprocessingsystem,
which can be integrated with our earlier work on signature verification and forgery
detection[2].

One of the most important tasks in automatic bank cheque processing is the
extractionofhandwrittensignaturesfrombankchequesandthenfeedingthemtoan
off-line signature verification system which tests the signature for authenticity. The
extraction and recognition of handwritten information from a bank cheque pose a
formidable task [3] which involves several subtasks such as extraction and
recognitionofsignatures,courtesyamount,legalamount,payeeanddate(seeFig.1).
Inoneofthemostpioneeringworksinthisfield,Djezirietal.[4]tackledtheproblem
ofextractinghandwritteninformationbymeansofanintuitiveapproachthatisclose
to human visual perception, defining a topological criterion specific to handwritten
lineswhichtheytermedasfiliformity.Theyextractedseveralsignaturesfromcheques
withpatternedbackgroundsusingthisfiliformitycriterion.

Banks Name
Checks Date
Banks Logo
Payees Name
Courtesy Amount

Legal Amount
in text
Signature
Bank code and
account number

Fig.1.DifferentfieldsinabankCheque

Thenatureofbankchequesisvariedandcomplexandthismakestheproblemof
automatic bank cheque processing very difficult [5]. The only way to extract
signatures from bank cheques and other forms is to have some sort of prior
informationaboutthelayoutofthedocument[6,7].Inthispaper,wehaveproposeda
similar technique to approximate the area of segmentation before extracting the
signaturesfromtheregionofinterest.

592

Proc. VIIth Digital Image Computing: Techniques and Applications, Sun C., Talbot H., Ourselin S. and Adriaansen T. (Eds.), 10-12 Dec. 2003, Sydney

SystemOverview

TheoverallblockdiagramofoursystemisgiveninFigure2.Thefirststepistoscan
the cheque or the form from which signature is to extracted and apply some preprocessingstepstoremovethenoise.Thesystemthenusesaprioriknowledgeabout
thedocumenttodeterminetheapproximateareaof interest.Thisisfollowedbythe
applicationoftheslidingwindowtechniqueinordertoextractthesignatureorother
userenteredcomponents.Thefollowingsectionsdescribetheindividualstagesofthe
systeminmoredetailforthecompleteunderstandingofthesystem.

Scanned Image

Pre-processing Techniques

Layout Analysis (Area

Approximation)

Sliding Window Technique

Automatic Cropping of the

Region of Interest

Extraction of other fields

such as Courtesy Amount,
Date etc.

Extraction of Signature

Recognition and Analysis

Signature Verification

Reference Database of
Signatures

Fig.2.BlockDiagram

Pre-processing

Binarizationisanimportantstepinbankchequeprocessing.Thisinvolvesremoving
the background and extracting useful information. For this task, we have used a BSpline filter which employs a threshold value to segregate the background from the
printedandhandwritteninformation.

593

Proc. VIIth Digital Image Computing: Techniques and Applications, Sun C., Talbot H., Ourselin S. and Adriaansen T. (Eds.), 10-12 Dec. 2003, Sydney

Theguidelinesonwhichtheuserhastofillinarepresentalloverthecheque.The
guidelineorbaselineremovalisthenextimportantpre-processingstep.Sinceweare
interestedinspecificfieldsonthecheque,bysegmentingthesefieldswecandoaway
withtheguidelines.Thuswedonotresorttoanyprocessingfortheirremoval.There
areotherpre-processingstepsasslantremovalofthedocumentandenhancementof
noisyimageswhichareappliedtospecificexamplesincasetheneedarises.

Extractionprocedure

Thisisthemostimportantmoduleintheentiresystem.Toperformthisprocedure,the
textimageshouldhavebeenpre-processingasexplainedintheearliersection.Only
then, signature segmentation can be achieved as all the noise and other extraneous
featureshavebeenphasedout.

Approximation area
(130,35,200,55)

Fig.3.Approximationarea

This methodcanbeusedtolocateaboxincheques.Inthis,asliding windowis

createdtomovehorizontallyfromlefttorightontheapproximationarea(referFig.4).
Thewidthofthewindowisfixedtoacertainnumberofpixelsandtheheightofthe
pixel will be set according to the height of the approximation area. As the sliding
window moves by one pixel at a time, the density of the pixels within the current
windowiscalculated.Thisdensityisusedtocalculatetheentropyasfollows:

E = P(a j ) log P(a j ) (1)

where P ( a j ) isthepixeldensity

594

Proc. VIIth Digital Image Computing: Techniques and Applications, Sun C., Talbot H., Ourselin S. and Adriaansen T. (Eds.), 10-12 Dec. 2003, Sydney

The entropy is a better choice than density because it introduces larger range of
valuesleadingtoeasierandmoreaccuratesegmentation.Theleftandrightbordersof
aboxaredeterminedbythetwomaximumvaluesofpixelentropy.Togetthetopand
bottombordersofthebox,weruntheslidingwindowverticallyfromtoptobottomof
the approximation area of the box by repeating the process of horizontal sliding as
showninFig.4.

0
1015

Widthofslidingwindowis2pixelswide

Horizontalslidingwindow
20
20 is the highest density therefore this window will
marktheleftborderofthebox.

Fig.4.SlidingWindowConcept

4.1 CropMethod
Thecropmethodasshowninfigure5isappliedonthedefinedapproximationareain
acheque.Itsobjectiveistolocatearectangularboxaroundanobjectofinterestand
removeotherobjectsoutsidethisarea.Ifthesignatureistheobjectofinterestinthe
cheque, this could be easily done. The crop method works by moving four vectors
fromfourdifferentdirections(namelyup,right,bottomandleft)towardstheobjectof
interest.Thisprocedureisillustratedasbelow:

Vector

Vector

Vector

Vector

Fig.5.Thecropmethod

595

Proc. VIIth Digital Image Computing: Techniques and Applications, Sun C., Talbot H., Ourselin S. and Adriaansen T. (Eds.), 10-12 Dec. 2003, Sydney

For each vector, it will stop moving when it finds a point (black pixel) in its
direction. Each vector will mark the border of each side of the rectangular box (i.e.
Vectortopmarksthetopborderoftherectangle,vectorrightmarkstherightborder,
vectorbottommarksthebottomborderandvectorleftmarkstheleftborder.)
4.1.1 Implementationonthesignature
Theoriginalchequeimagecanbeincolourorgrayscale.Thescannedimagemustbe
convertedtobinaryformatwiththehelpofanythinningalgorithm.Wehaveusedthe
modifiedSPTAasoutlinedin[2].Onetheapproximationareaofthesignaturefora
particularcheckisdefined,thecropmethodisthenappliedtotheapproximationarea.
Thecroppedimagecontainsonlythesignaturereadytobefedtoanofflinesignature
verificationsystem.
4.1.2 Implementationonthecourtesyamountbox
Inthiscase,theapproximationareaofthecourtesyamountboxiscalculatedfroma
virgin model of the cheque and this information is pre-defined to the system. The
widthofslidingwindowissettotwoorthreepixelsdependingonthethicknessofthe
box.Forathickerbox,wewilluseasizeofthree.Slidingwindowisappliedtothe
approximationarea.Thenwewillremovetwopixelsfromeachborderoftheboxto
remove the box lines to get only the content of the box, i.e., the courtesy amount
itself.

Fig.6.Theextractedcourtesyamountbox

Wewillthenapplytheslidingwindowmethodagaintotheresultingimage(with
boxremoved).Butthistimewedothesegmentationbasedontheminimumentropy
ofthewindow,becausethisvaluerepresentsthegapbetweencharacters/digits.The
segmentedcharacterswillbefedtothefuzzysystemforrecognitionandverification.

ExperimentalResults

The present system for extracting signatures from bank cheques and other different
kindsofformshasbeenimplementedonaPentiumIIIpersonalcomputerunderthe
Windowsenvironment.TheentirecodehasbeenwritteninVisualBasic.Theimages
werescannedusingaHPScanjetwith600dpiresolution.Ittooklessthanaminuteto
extractasignaturefromabankcheque,oncetheareaapproximationhasbeendoneon
thefirstmodelofthecheque.

Oursystemhasachievedatotalaccurateextractionrateof99.26%onadatabaseof
211images.Theextractionratesofsignaturesfromdifferenttypesofdocumentsare

596

Proc. VIIth Digital Image Computing: Techniques and Applications, Sun C., Talbot H., Ourselin S. and Adriaansen T. (Eds.), 10-12 Dec. 2003, Sydney

summarized in Table 1. The system failed to completely extract only one signature
from a bank cheque. This was due to the fact that the person who had signed the
chequehadnotdonesointhestipulatedareaandhencethesystemcouldnotcorrectly
approximatetheareatobesegmented.

Table1.Overallresults

Typeof
Document

BankCheques

TutorForms

DonationReceipts

Number

CorrectExtraction
(%)

Partial/NoExtraction
(%)

45

97.78

2.22

130

36

100

100

In the following sections, we describe in detail two typical types of extraction

problems to which we have successfully applied sliding window technique. These
experiments have been conducted in order to show the easy adaptability and
robustnessofour system.Thefirstinvolvedextractionof signaturesofatutorfrom
student assessment forms collected over a period of three months. The second
example,whichwedescribeinmoredetail,hasusedfuzzyenhancementtechniquein
itspre-processingstageasthedocumentwashighlycorruptedinnature.Ourmethod
canalsobeappliedtoseveraldifferentformstoextractnotonlysignaturesbutother
userenteredcomponentsprovidedtheapproximateareaofinterestispre-specified.

5.1 ExampleI:Signatureextractionfromapracticalexamform
In this experiment, around 130 practical assessment forms were collected from the
studentsofanElectricalEngineeringlabcourseoveraperiodof13weeks(duration
of a normal academic semester) at the University of Queensland. The aim was to
automaticallyextractthetutorsignaturefromtheformandcheckforitsauthenticity.
Theformisacomplicateddocument witha numberofboxesandfields markedfor
entering data. The region of interest, in this case, the box containing the tutor
signaturewasapproximatedandtheinformationsoobtainedwasusedforextracting
thesignatures.Thecompleteprocessisasshowinthefollowingfigure.

597

Proc. VIIth Digital Image Computing: Techniques and Applications, Sun C., Talbot H., Ourselin S. and Adriaansen T. (Eds.), 10-12 Dec. 2003, Sydney

Fig.7.Procedureshowingtheextractionoftutorsignatures

598

Proc. VIIth Digital Image Computing: Techniques and Applications, Sun C., Talbot H., Ourselin S. and Adriaansen T. (Eds.), 10-12 Dec. 2003, Sydney

5.2 ExampleII:Signatureextractionfromthecarboncopiesofreceipts

In addition to extracting signatures from bank cheques and tutor forms, we have
attemptedamorechallengingtaskofextractingsignaturesfromthescannedimages
ofcarboncopiesofdonationreceiptsofHeritageBuildingSociety.Thereceiptswere
signed over a period of more than three years and the carbon copies of these
documents were hazy and not clear in content. In order to remove the high noise
contentposedbythedamageddocumentsandalsotoimprovetheappearanceofthe
signatures to be extracted before applying the sliding window technique, we have
applied a fuzzy enhancement method [8] which enhances the overall quality of the
documents. The signatures which are extracted using this technique are perfectly
suitableforsupplyingtoanyoff-linesignatureverificationsystem.Figure8showsthe
original noisy document before enhancement is done and the enhanced signature
whichisthenextracted.

Fig.8.Fuzzyenhancementofsignaturesbeforeapproximationisdone

599

Proc. VIIth Digital Image Computing: Techniques and Applications, Sun C., Talbot H., Ourselin S. and Adriaansen T. (Eds.), 10-12 Dec. 2003, Sydney

Conclusions

The first and foremost step towards developing a complete bank cheque and form
processing system is to automatically extract the handwritten signatures and other
userenteredinformationfromthedocument.Although,thissoundssimplebutisquite
difficult due to the problem posed by varied backgrounds and the intermixing of
machineprintedandhandwritteninformation.Theonlywaytotacklethisproblemis
to seek some priori information about the location of the signature and other
importantfieldsofinterest.

Thispaperisastepinthatdirectionasitpresentsanautomaticsignatureextraction
system based on a priori information of the layout of the document. The proposed
sliding window is able to extract signatures from any type of documents, as was
shown on two examples. In further work, we plan to integrate this system with
signature verification so that the whole process of bank cheque authentication is
automated.

References
1. Holland,T.,G.,ChecksandBalances.SecurityManagement.43(1999)76-82.
2. Hanmandlu, M., Yusof, M., H., M., Madasu, V., K., Off-line signature
verificationandforgerydetectionsystembasedonstructuralparameters.Pattern
Recognition.Submittedforreview.
3. Suen, C.Y., Xu, Q., Lam, L., Automatic recognition of handwritten data on
chequesFactorFiction?PatternRecognitionLetters.20(1999)1287-1295.
4. Djeziri, S., Nouboud, F., Plamondon, R., Extraction of signatures from check
background based on a filiformity criterion. IEEE Transactions on Image
Processing.7(1998)1425-1438.
5. Dimuaro, G., Impedovo, S., Pirlo, G., Salzo, A., Automatic Bankcheck
Processing: A New Engineered System. International Journal of Pattern
RecognitionandArtificialIntelligence.11(1997)467-504.
6. Liu,K.,Suen,C.Y.,Cheriet,M.,Said,J.N.,Nadal,C.,Tang,Y.,Y.,Automatic
Extraction of Baselines and Data from Check Images. International Journal of
PatternRecognitionandArtificialIntelligence.11(1997)675-697.
7. Okada, M., Shridhar, M., Extraction of User Entered Components from a
Personal Bankcheck using Morphological Subtraction. International Journal of
PatternRecognitionandArtificialIntelligence.11(1997)699-715.
8. Vijayaprasad, P., Elsid, A.G., Hanmandlu, M., Enhancement of Fingerprint
Image A Fuzzy Approach. International Conference on Software,
Telecommunications and Computer Networks (SoftCOM 2003). Accepted for
presentation.

600